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Wire-Tap Channel Il 


By L. H. OZAROW and A. D. WYNER* 
(Manuscript received April 2, 1984) 


Consider the following situation. K data bits are to be encoded into N> K 
bits and transmitted over a noiseless channel. An intruder can observe a subset 
of his choice of size u < N. The encoder is to be designed to maximize the 
intruder’s uncertainty about the data given his yu intercepted channel bits, 
subject to the condition that the intended receiver can recover the K data bits 
perfectly from the N channel bits. The optimal trade-offs among the param- 
eters K, N, and yw and the intruder’s uncertainty H (H is the “conditional 
entropy” of the data given the yu intercepted channel bits) were found. In 
particular, it was shown that for py = N — K, a system exists with H ~ K — 1. 
Thus, for example, when N = 2K and u = K, it is possible to encode the K 
data bits into 2K channel bits, so that by looking at any K channel bits, the 
intruder obtains no more than one bit of the data. 


I. INTRODUCTION 


In this paper we study a communication system in which an unau- 
thorized intruder is able to intercept a subset of the transmitted 
symbols, and it is desired to maximize the intruder’s uncertainty about 
the data without the use of an encryption key (either public or private). 

Specifically, the encoder associates with the K-bit binary data 
sequence S¥* an N-bit binary “transmitted” sequence X%, where N > 
K. It is required that a decoder can correctly obtain S* with high 
probability by examining X%. The intruder can examine a subset of 
his choice of size u of the N positions in X¥, and the system designer’s 
task is to make the intruder’s equivocation (uncertainty) about the 


* Authors are employees of AT&T Bell Laboratories. 
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data as large as possible. The encoder is allowed to introduce random- 
ness into the transformation S* — X¥, but we make the assumption 
that the decoder and the intruder must share any information about 
the encoding and the randomness. This assumption precludes the use 
of “key” cryptography, where the decoder has the exclusive possession 
of certain information. 

As an example, suppose that K = 1, N = 2, and p = 1. Let the data 
bit be S, and let & be a uniform binary random variable that is 
independent of S. Let X? = (£, & ® S), where ® denotes modulo 2 
addition. If the intruder looks at either coordinate of X?, he gains no 
information about S, so that the system has perfect secrecy. The 
decoder, however, can obtain S by adding (modulo 2) the two compo- 
nents of X?. 

Our problem is to replicate this type of performance with large K, 
N, and u. In fact, we assume that K ~ RN and p = aN, where R and 
a are held fixed and N becomes large. Roughly speaking, we show that 
perfect secrecy is attainable provided that p is not too large, specifically 
w=sN-Koras1-—R. In Section II we give a precise statement 
and discussion of our problem and results, leaving the proofs for 
Sections III through V. 

This problem is similar to the wire-tap channel problem studied in 
Ref. 1. A special case of the problem studied there allows an intruder 
to examine a subset of the encoder symbols that is chosen at random 
by nature. In the present problem, the system designer must make the 
system secure against a more powerful intruder who can select which 
subset to examine. 


Il. FORMAL STATEMENT OF THE PROBLEM AND RESULTS 


In this section we give a precise statement of our problem and state 
the results. 

First a word about notation. Let Y be an arbitrary finite set. Denote 
its cardinality by | Z|. Consider 2, the set of N-vectors with 
components in %. The members of ZN will be written as 


uv = (ui, U2, **", un); 


where subscripted letters denote components and boldface super- 
scripted letters denote vectors. A similar convention applies to random 
vectors, which are denoted by uppercase letters. When the dimension 
of a vector is clear from the context, we omit the superscript. Finally, 
for random variables X, Y, Z, etc., the notation H(X), H(X|Y), 
I(X;Y), etc., denotes the standard information theoretic quantities 
as defined, for example, in Ref. 2. 
We now turn to the description of the communication system. 
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1. The source output is a sequence {S,}$, where the S;, are inde- 
pendent and identically distributed (i.i.d.) binary random variables 
with uniform distribution. 

2. The encoder with parameters (K, N) is a channel with input 
alphabet {0, 1}*, output alphabet {0, 1}‘, and transition probability 
gr(x%|s*). Let S* and X* be the input and output, respectively, of 
the encoder. 

3. The decoder is a mapping 


fo: {0, 1}% > {0, )*. 
Let S = ($1, So, -+- , Sx) = fp(X%). The error rate is 


Loe P 
P. — ie my Pr{S; ¥ Sp}. 
K p21 


4, An intruder with parameter u < N picks a subset © C {1, 2, 
..-- ,.N}, such that | _Y | = uw and is allowed to observe X,,n © X% Let 
ZN = (Z,, --- , Zn), defined by 


2, ES 


denote the intruder’s information. The system designer seeks to max- 
imize the equivocation 


AA min H(S*|Z%). 


3! S |= 


z= |X» neg 


Thus, the designer is assured that no matter what subset the 
intruder chooses, the intruder’s remaining uncertainty about the 
source vector is at least A. When A = K, the intruder obtains no 
information about the source, and the system has attained perfect 
secrecy. 

In this paper we study the trade-offs between K, N, A, and P,. As 
we shall see, it will be useful to consider the normalized qualities 
K/N, n/N, and A/K. Thus, K/N is the rate of the encoder equal to the 
number of data bits per encoded bit, n/N is the fraction of the encoded 
bits that the intruder is able to observe, and A/K is the normalized 
entropy. 

The intruder who observes Z can reconstruct the data sequence 
S* with a per-bit-error probability of, say, P!. It follows from Fano’s 
inequality that h(P?) = A/K, where h(-) is the binary entropy function 
defined below eq. (2). Thus, A/K = 1 implies that P; ~ 1/2, which is 
essentially perfect secrecy. 

We will say that the triple (R, a, 6) is achievable if for all « > 0 and 
all integers No > 0, there exists an encoder/decoder with parameters 
N=No, K2=(R—- ON, p= (a — ON, A= (6 — €)N, and P. S «. We 
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will show in the sequel that (R, a, 5) is achievable for 0 < R, a < 1, 
and 


1, O<sas<1-R, 


0<6<2 (1-2) a ee (1) 
R ’ th po 


A graph of the achievable (a, 6) pairs for fixed R is given in Fig. 1. 

The following theorem, a proof of which is given in Section III, is a 
“converse” result that gives a necessary condition on achievable 
codes. 


Theorem 1: If there exists a code with parameters (K, N, A, P.), then 


kK, OsusN-K, 
ae N-K<pE€N, (2) 


where h(A) = —A log A — (1 — A)log(1 — A) is the binary entropy 
function. 

Now if (R, a, 6) is achievable, for arbitrary « > 0, there must be an 
encoder/decoder with parameters N, K = (R — 6)N, » = (a — ON, 
A= (6— ON, P. < «. Applying Theorem 1 to this code yields 


1, 
< oes 
oa BH) + O16) + ho, 1-Rsat+Oe) S1,{’ 

which is (1) as e > 0. Thus conditions (1) are necessary for a triple to 
be achievable. Theorem 2, which is also proved in Section III, implies 
that (R, a, 6) is achievable if (1) is satisfied. 
Theorem 2: Let 1 —-R<a<1. Then, for all e>0, No = 1, there exist 
an N = No and an encoder/decoder with parameters K = RN, u = aN, 
A/K = [(1 — a)/R] — «, and P, = 0. 


6 





1-R 1 sd 
Fig. 1—Achievable (a, 6) for fixed R. 
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The idea behind the proof of Theorem 2 is the following. Parti- 
tion the set {0, 1}¥ into 2* subsets {A,,}?. with equal cardinality—i.e., 
|A,,| = 2"*. The 2% possible values of S* can be put in 1 to 1 
correspondence with these subsets. When S* corresponds to A,, (1 < 
m = 2*), the encoder output is uniformly distributed on A,,. Since the 
{Am} are disjoint, the decoder can recover S* perfectly and P, = 0. We 
show (by random coding) that there exists a partition satisfying 
Theorem 2. 

A convenient way to partition {0, 1}% is to let the sets {A,,} be the 
cosets of a group code G with N — K information symbols (so that G 
has 2* cosets). Theorem 3, which is proved in Section IV, asserts that, 
in fact, we can do quite well with codes of this type. 

Theorem 3: If the triple (R, a, 5) satisfies (1), then it is achievable using 
an encoder/decoder derived from a group code. 

The following simple lemma allows us to establish the achievability 
of all triples on the straight line of Fig. 1 connecting points A and B 
by proving only the achievability of point A. 

Lemma 1: Suppose that we are given an encoder/decoder fx, fp with 
parameters N, K, and P.. Suppose that there are two intruders that 
have parameters pw = py, We and A = Ay, As, respectively. Then, if 
Ho = 1, 

Ag = Ay — (u2 — 1). (3) 
Remark: Inequality (3) can be rewritten as 


= i o/N — wi/N 
(As/K) = (A,/K) eae ) 


from which we conclude that (R, a, 6;) is achievable implies that 
(R, a2, 52) is achievable where az => a; and 


i= - (25), 


In particular, if aj = 1 — R, 6, = 1, then 


62 = (1 _ ag)/R. 
Proof of Lemma 1: Let “A © &% € {1, 2, ---, N}, where |.“ = mw, 
| A| = ue. Let ZN correspond to “(i = 1, 2), iie., Zi = (Zia, ---, Zin) 
and 


za JX» GES 
i, FES, 


Then, 
H(S*|Z.) — H(S*|Z,) = H(S*| Zs, Z:) — H(S* |Z) 
= —1(S*; Z.|Z,) = —H(Z2|Zi) = — (ue — m1), 
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where the first equality follows from % C %. Thus 


H(S*|Z2) = H(S*| Zi) — (2 — 41) 
= Ai — (u2 — w1) (4) 
from the definition of A. Minimizing (4) over all %, with |%| = ue, 
yields (3) andthe lemma. O 


Finally, we state a theorem that is a rather surprising strengthening 
of Theorem 2. Its proof is given in Section V. 


Theorem 4: For arbitrary K, N (1s Ks N), anduw=N — K, there 
exists an encoder/decoder with P, = 0 and 


228 
YN 





A2zK-1- 


Ill. PROOF OF THEOREMS 1 AND 2 


Assume that S¥, X™, Z™ and S correspond to a source/encoder/ 
decoder as defined in Section II, with parameters K, N, A, and 
P.. Then, making repeated use of the identity H(U, V) = H(U) 
+ H(V|U), we obtain 


A = H(S*|Z) = H(S, Z) — H(Z) 
= H(S, X, Z) — H(X|S, Z) — H(Z) 
= H(S|X, Z) + H(X, Z) — H(X|S, Z) — H(Z) 
= H(S|X, Z) + H(X|Z) — A(X|S, Z). (5) 


Now 


H(S|X, Z) = H(S|X, Z, 8) = H(S|S) 
< Kh(P.), 


where the last inequality follows from Fano’s inequality (see Ref. 2). 
Also, since H(X|Z) is the entropy of those N — yu coordinates of X 
not specified by Z, we have H(X|Z) <= N — uy. Finally, noting that 
H(X|S, Z) = 0, we have from (5) 


AsN-yu+ KXP.), 


which is Theorem 1. 

We now give a proof of Theorem 2, which proceeds along the lines 
suggested in Section II. Let K, N be given, and let {A,,}, 1 < m s 2%, 
be a partition of {0, 1}% into subsets A, C {0, 1}% such that |A,| = 
QN-K’ As in Section II, the partition defines a code: to encode message 
m(1 sm Ss 2), we let X™ be a randomly chosen vector in A,,. Since 
the A,, are disjoint, P, = 0 and H(S|X, Z) = 0. Further, since the 2* 
messages are equally likely and |A,,| = 2%~*, X is uniformly distrib- 
uted on {0, 1}%, so that its coordinates are i.i.d. uniform binary random 


2140 TECHNICAL JOURNAL, DECEMBER 1984 


variables. Thus H(X%|Z%) = N — p. We conclude from (5) that for 
this encoder 


A=N-4yu- H(X*%|S%, Z). (6) 

Now let z € {0, 1, ?}% be a possible value for the intruder’s infor- 

mation, and let x € {0, 1}%. We say that z is “consistent” with x if z 

can be obtained from x by changing a subset of the coordinates of x 

to ?’s. Next, let L = 1 be an integer to be chosen later. We say that a 

partition {A,,} is “good” if for all m(1 < m s 2*) and all z € {0, 1, ?}% 
with exactly N — n?’s, 


card{x € A,: z is consistent with x} < L. 
If our encoder corresponds to a good partition for some L, then 
H(X*|S%, Z%) < log L, 


and (6) yields 
A=>N-uz-— log L. (7) 
At the end of this section we will prove the following proposition 
about the existence of good partitions. This will lead us directly to 
Theorem 2. 
Lemma 2: Let K, N, and wu be such that 


N-p-K<0. (8) 


Then, there exists a good partition (with parameters K, N, and p) 

provided 

2N + K + 2 loge 
K+p-N © 


Now let R, a, e, and No be given as in the hypothesis of Theorem 2. 
Then, using 2 log e < 3, we write for N = 1, 


N+K+2loge_ 1+hk+3 
K+p-N ~~ a-(1-R) 
Thus, there exists a good partition with L < B + 1, and we con- 
clude from (7) that there exists a code with A/K = [(1 — a)/R] — 
{flog(B + 1)]/(RN)}. If we choose N = No, «R/log(B + 1), then the 
existence of this code establishes Theorem 2. It remains to prove 
Lemma 2. 
Proof of Lemma 2: Let {A,}, 1 < m s 2*, be a partition of {0, 1}4, 
where |A,| = 2%-*. Let W(Ai, ---, Aox) = 0 or 1, depending on 
whether {A,,} is good. We write 


L> (9) 


AB<o, 


oK 


W(A, eae Aok) = > 2 (An; Z), (10) 


m=1 z 
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where the inner sum is taken over all z € {0, 1, ?}* with exactly 
N — p?’s, and ¢(An, z) = 1if 


card{x € A,,: z is consistent with x} = L, 


and ¢(A,,, z) = 0 otherwise. 

We now choose the partition at random with uniform distribution 
on the set of partitions of {0, 1}% into 2* classes of equal size. The 
expectation EW satisfies 


EW < ¥ Y E®(An, Z). (11) 


The expectation in the right member of (11) is taken, as indicated, 
with z held fixed. Let us define the following quantities: 


Q(z) = {x C {0, 1}%: x is consistent with z}, 
ny = | Q(z) |= 2%, 
n = |{0, 1})%] = 24, 
P= Ag =o". (12) 


We now compute E®(A,,, z). The r members of A, are chosen at 


random from {0, 1}. (without replacement). The probability that 
exactly t members of A,, belong to Q(z) is 


CNL) 
() 


To see this, observe that there are (") ways to choose the set A,,. 


The ¢ members of A,, that belong to Q(z) can be chosen in ce ways, 


and the remaining (r — t) members of A,, can be chosen from the 


complement of Q(z) in : _ ‘g ways. 


~ Now 
n\in—- ny ny n 
(ran), heed 
1 = —.— = —_x—.. 
n n 
(7) (7) 
Also, using i < n{/t!, and 
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i : ) n! ri(n — r)! 
(") ~(n—-rtoOr—-t! al 
r 





= r(r— 1)(r — 2)---(r —t# + 1) 
~~ (ntt—r\(nt+t—r—1)---(n—-rt]) 
r’ (r/n)‘ 


“(ant arn)" 


we have 
(n,r/n)‘ 
tll — r/n)t’ 
Thus 
Qn-K oo) t 
(mr/n) 
E®(A,,, Z) = < ———_—_——_., 
( 2 2 ms 2 t!(1 — r/n)' 
Using (12), we have (nmr/n) = 2%**, (1 — r/n) = 1/2, so that 
00 9t 
E®(A,n, z) < 3 9(N-u-K)f ae 
t=L t! 
< 9(N-u-K)L y at Q(N-H-K)L 92 
= t=o Ut! 


Substituting into (11) yields 
Ev < > Sy 9(N-n-K)L+2 loge 


m Zz 
< Q(N-u-K)L+2loget+K+2N 


If L satisfies (9), then EW < 1. Since W is integer valued, there must 
exist a particular partition, say {A>} such that W(AT, --- , Asx) = 0. 
This is our good partition. O 


IV. GROUP CODES AND THEOREM 3 


In Sections II and III, we discussed how to construct encoder/ 
decoders based on a partition {A,,} of {0, 1}. In this section we consider 
the special case where the partition {A,,} is defined by a group code 
and its cosets. 

Let H be a K X N parity-check matrix, which we assume has rank 
K. Let the partition {A,,}, 1 < m < 2%, be the code defined by H and 
its cosets. Thus |A,,| = 2%-*, for 1 <s m < 2%. To encode message 
s = (s,, --- , Sx), the encoder makes a random selection of one of the 
2N-K members of the A, corresponding to s. This is equivalent to 
letting X% be a random choice from the 2" solutions of 
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HXt = s', (13) 


where {+ denotes matrix transpose. Note that since S is uniformly 
distributed on {0, 1}*, X% is uniformly distributed on {0, 1}%, and its 
coordinates X,, Xo, --- , Xy are i.i.d. uniform binary random variables. 
The decoder observes X* and computes HX", which is the message. 
Thus P, = 0. We now show how to compute A in terms of certain 
distance-like properties of the parity-check matrix. 
Definition: Let C,, Co, --- , Cy be the columns of H (C, is a K-vector). 
Let Y € {1, 2, --- , N} and define D(_“) to be the dimension of the 
subspace spanned by {C,}, n € “% For a given K X N parity-check 
matrix H, define for0 < » < N, 
D*(u) = min D(“). (14) 


S |=N-n 


We now state Lemma 3. 


Lemma 3: Let D*() correspond to the K X N parity-check matrix H. 
Let w, w’ be the minimum weight of the code and dual code, respectively, 
defined by H. Then, (a) for N-~wt+1<sypSN, D*(p) = N — p; (0) 
forOsyusw’-1,D*(n) =K 

Proof of Lemma 3: Assertion (a) follows immediately on observing 
that all sets of w — 1 columns of H are linearly independent. Thus 
D(Y)=|S|,for|AY|sw-1lifN-wt+1syusQN, then N —- 
pe <= w — 1, so that 


D*(uy) = min D(iY)=N-4g, 
| |=N-u 


which is assertion (a). ; 
Now assertion (b) states that all submatrices H = (C,, C;,, 
, C;,) of H have rank K when q = N — w’ + 1. To establish this 
savertiou: assume that rank H < K. Then there exists a set of linear 
row manipulations which transform H into a matrix with a row of 
zeros. These same row manipulations will transform H into a matrix 
for which a row has weight < N — gq. Since the dual code is the row 
space of H, N-—q2=w’ org Ss N — w’, establishing assertion (b). O 
We now give Lemma 4. 
Lemma 4: When an encoder/decoder is constructed to correspond to the 
parity-check matrix H, then 


A = D*(u). (15) 


Proof of Lemma 4: Let S, X, Z correspond to an encoder/decoder with 
parameters K, N, A and (P, = 0), derived, as discussed above, from a 
parity-check matrix H = (Ci, ---, Cy). Since P. = 0 and X% is 
uniformly distributed on {0, 1}, eq. (6) applies. Thus, Lemma 4 will 
be established when we show that 
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H(X*|S*, Z™) = N- p — D*(u). (16) 


Now suppose that S“ = s and ZN = z. Without loss of generality, 
assume that the last » coordinates of z are copies of the corresponding 
coordinates of X. Thus, given S* = s, Z" = z, the remaining unknown 


coordinates of X are precisely the solutions for x;, --- , xv_-, of 
N-u N 
Y Cax,=s'+ Y Cx, Aa. (17) 
n=1 n=N~p+1 

Since the number of solutions is N — w — rank (C;, --- , Cy_,), and, 


given S = s, Z = z, all these solutions are equally likely, then (16) 
follows; hence the lemma. O 

Before continuing with the proof of Theorem 3, we digress to apply 
Lemma 4 in an example. Let K = 4, N = 8, and construct an encoder/ 
decoder using the self-dual Hamming code with block length 8 and 
four information digits and four check digits. Then, w = w’ = 4, so 


that 
4= Kk, Osyps 3, 
A= D*(p) = 43, w= 4, 
Ss yu 


Np; 


Thus, the encoder/decoder is optimal for all » except u = 4, when A 
is only one bit less than ideal. 

We will establish Theorem 3 via a random code argument. Towards 
this end, we establish the following lemmas. 
Lemma 5: Let 1 < m <n, and let the m X n matrix A over GF(2) be 
chosen at random with uniform distribution on the set of 2” binary 
m X n matrices. Then, forl1 <= L<sm, 


Pr{rank A < m — L} s Q0@tv@rmsn, 


Proof of Lemma 5: Let us choose the n columns of A sequentially and 
independently. Let d(j) be the dimension of the linear space spanned 
by the first 7 columns. Suppose that d(j) = k s m. With probability 
Q*-™ d(j + 1) =k; and with probability (1 — 2"), dj +1) =k+1. 
This sequential choice of the columns is modeled by the Markov chain 
of Fig. 2. 


1-2 1-21 1-271 


Q-m 2i-m ; 22m 2-1 1 


Fig. 2—Markov chain used in proof of Lemma 5. 
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Begin at state 0. With each choice of a column, advance one state if 
and only if this choice increases the dimension of the space spanned 
by the columns chosen so far. The rank of the matrix A is d(n) and is 
equal to the state at which we find ourselves after all n columns are 
chosen. Let I'(k) denote the set of paths z that start at state 0 and 
terminate at state k(0 < k <= m). Then 


m—-L-1 
Prfrank A<m-L}= DY Y Prix}. (18) 
k=0  w€T(k) 


Now let the path a € I(k). This path contains exactly n — k self- 
loops, each of which has probability < 2~"**. Thus, for x € I'(k), 


Prfx} < 2-mthe-®, 


Also, since | (Rk) | = (r) eq. (18) yields 


m—-L-1 
Prirank A<m-L}s Y (7) g-(m—k)(n-k) 
k=0 
Since the exponent is nondecreasing in k(k = m < n), we have 
m-L-1 
Prirank A<m-—L<s Y (7) g-(L+1)(n—m+L+1) 
k=0 
< 9° geen 


which is Lemma 5. OF 


Lemma 6: Let 1 <= m <n, and let the m X n matrix A over GF(2) be 
chosen at random with uniform distribution on the set of 2” binary 
m X n matrices. Then 


m-1 


Pr{rank A = m} = [J (1 -— 277”) 
j=0 


ee —mgzm-i-n SA ae mgm-i-n 
= p 1 = gm-i-n = Te gm-i-n 2 
Proof of Lemma 6: Choose the rows of A sequentially. As in the proof 


of Lemma 5, the probability that the dimension of the space spanned 
by the first j rows is equal to is 


(1 = 2-")\(1 Lae gett). ‘ (1 = yaa ia 0 


The rest of the lemma follows from In(1 — u) = —u/(1 — wu) and 
e“>1-—u. 

We now turn to Theorem 3. Let R > 0 be given and held fixed. We 
will show that 6 = 1 and a = 1 — R is achievable, and the remainder 
of the theorem will follow from Lemma 1. Let « > 0 be arbitrary. We 
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will show that there exists an encoder/decoder with parameters N, 
K= RN,» = (1 -—R- ON, and A= K — L, provided that 


L = 3/e. (19) 


We proceed as follows. Let H be a K X N parity-check matrix, and 
let L satisfy (19). Let D*(u) correspond to H, and define 


_ fi, D*(u) << K—-—L or rank(A) < Kk, 
ne ie otherwise. (20) 
We must show that there exists an H with ®(H) = 0. We can write 
WA)s YY  &(H, Y) + $(A), (21a) 
SC{L,---,N] 
1 l=n 
where 
_ ji, rank(H) < K, 
Poll) = 1¢ otherwise, (21b) 
and 
=, Wag D(iY)<K-L, 
oH, 7) = ie otherwise. (21¢) 
If we choose H = (Cy, --- , Cy) at random with uniform distribution 
on the set of 2%" binary K x N matrices, then (21) yields 
EW(H)s % E®(H, Y) + E%(H). (22) 
|Yl=n 
Let Y with || = u, be arbitrary, and let A = (C;,C;,---Ci,_), 
where “= {i,, -+- , in-,}. Then, ®(H, “) = 1 if and only if rank A < 


K-—L, and E®(H, “) = Pr{rank A < K — L}. We can apply Lemma 
5 with n = N — p, m= K, to obtain 
E@(H, SY) s QV H+ Wa), (23) 
Similarly, we can apply Lemma 6 with A = H, n = N, and m = K, to 
obtain 
Kgk-N-1 KQk-N 
E®)(H) s ] — gk-n-i = ] — ohn: (24) 


Since there are no more than 2” subsets “% (22) through (24) yield 
(using N-yp-K=c«N, K=RN) 


Ke 
Ev( H) < Q-(E+1)(N-n—-K)+(N-p) +N Bi = oer = 5IN 
R N27 G-R)N 
< Q-LeN+2N + To o-BN . (25) 
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Since L satisfies (19), the first term in the right member of (25) is less 
than 1/2. Furthermore, for N sufficiently large, the second term in 
(25) is also less than 1/2. Thus 


EW(H) <1. 


Since W(.) is an integer-valued function, there must exist a K X N 
matrix Hy such that W(Ho) = 0, so that rank Hp = K and for the 
corresponding encoder/decoder, A = D*(y) = K — L, which is what we 
set out to prove. Thus, we have shown that for arbitrary R > 0, the 
triples (R, a, 6), where a = 1 — R, 6 < 1, are achievable, completing 
the proof of Theorem 3. 


V. PROOF OF THEOREM 4 


We restate Theorem 4 here: 
Theorem 4: For all K, N for which 1 s K <= N, and » = N — K there 
exists an encoder/decoder pair for which error-free decoding is possible, 
and 


2:23 

Ue’ 
In particular, as K and N grow at a fixed ratio (i.e., K = RN as 
N-— ©), A can be made as close as we like to K — 1. 

We shall use a random coding argument to prove Theorem 4. A code 
for this problem is a partition of all 2" sequences into 2* message 
“bins,” each of size 2’-*. Each message is transmitted by randomly 
choosing an element of the bin corresponding to the desired message, 
with all elements chosen equally likely. The ensemble of codes is the 
set of all partitions, chosen equally likely. 

We shall obtain a lower bound on A as a function of the code selected 
and then show that the average (over the ensemble of codes) of A for 
any subset of pu bits selected is greater than K — 1. We then obtain 
the variance (again, over the ensemble of codes) of the bound. Using 
the variance, we can invoke Chebyshev’s inequality to bound the 
probability that the code yields A < K — 1 — « for any positive «. Since 
the probability distribution over the code ensemble is uniform, this 
bound is the fraction of the number of all possible codes that fail to 
provide acceptable secrecy for the » bits selected. Since there are a 





Az=k-1- 


limited number of sets of y bits (a) to be precise], the total number 


of codes that fail to provide acceptable secrecy for any choice of p bits 
can be bounded. The condition that the total number of such codes 
must be less than the number of codes in the ensemble, which guar- 
antees the existence of a good code, yields the theorem. 
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While the proof as just outlined is conceptually simple, the mechan- 
ics (primarily the variance computation) are complicated, and are 
outlined in the Appendix. 

We proceed: Let © be an index set (of size N — K), which is a 
subset of {1, 2, --- , N}. Let Z be the observed vector resulting from 
observations of the positions of transmitted code word X with indices 
in Y and let S be the message (we omit superscripts). Now 


A 4 H(S|Z) = H(S) -— H(Z) + H(Z|S) (26) 
follows from applying the identities 
H(A, B) = H(B, A) = H(B|A) + H(A). 


By the construction of the code, all values of X are equally likely, so 
that all values of Z are as well, and (26) becomes 


A=K-(N-K)+H(Z|S). 
Let n(z, s) be the number of code words corresponding to message 
s that are consistent with z, for each s and z. Then, 


Pr[Z = z|S = s] = ee, 


since the total number of code words corresponding to s is 24~*, 
Therefore, 


H(Z|S) =— ¥ ¥ plz, s)logep(z|s) 


=- 03 SEP piolos, See 
= sy DD nle, s)[N — K — logn(z, s)]. 








Clearly, 
¥ n(z, s) = 2N-* ¥ P(Z = z|S =s) 


— 9N-K 
and the number of s is 2%, so 
H(Z|S) = N - K-35 EE nz, s)logsn(z, 8), 
yielding 


A=K- = ¥. ¥ n(z, s)logen(z, s). 
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We will lower bound A by using 


1 
logex = in 2 In x 


1 x 
= ty (mE+ine) 


1 [x 
<i (E-1+ma), 


which is true for any positive a. Therefore, 





A ee = a pg UL rte ye 2 -1+inal. 
Since we have just shown that 
Yd n(z, s) = 2%, 
then 
= 1 ve s) 
a Pa in ONin 9 & > 


Interpreted as a function of the randomly selected code, A; is a random 
variable. The expectation n?(z, s) is constant for all z and s by the 
symmetry of the code selection, and 








1 n’@,s) 


* 
= K + —— — logea — ino 


In 


This bound is maximized by using a = n7(z, s), which yields 


A, = K — logen*(z, s). 
The variance of A; can be written as 
Var(As) = Bl(As — 5)" = sana5 2 2 mes) — RP (on 
Z 7 2 n2 


Denote the four generic (z, s) pairs by (1, 1) (1, 2), (2, 1), and (2, 2). 
Also let 


A= 2%, 

B= 2K, 
and 

Ceo, 


Equation (27) can be written as 
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1 n?(1, 1) — (n?)?? 
Var(As) = 7 nt {connor 


— (n2)\2 
246=%) 1)n?(2, 1) = (n*) | 
(n*)? 
n2(1, 1)n?(1, 2) — (n°)? 
+ (B — 1) (ez) | 


+ (B-1)(C - 1) eee Dae 2 a 
(n?)? 
or 
— 
(n?)?A In?2 
Using the bounds on M,, Mo, M3, and M, given in the Appendix gives 
1 1 
A In?2 (1 + T,)? 
-[5T, + 5T7 + Ti — 27, — 47? — 273 + T, + 473] 
= eee aa [87 + 5Ti — 7); 


where 7, (defined in the Appendix) is less than or equal to one. 
The right-hand side is maximized (over the allowable range of T,) by 
T; _ 1, so that 


Var(A,) = [M, + M, + M; + M4]. (28) 


Var(A,) = 


3 6.244 

MOSS ae 
6.244 

—"gN- 


The mean of A, is 
A, = K — logan’. 
Since (see the Appendix) 


gg tee 1) 2 2, 


then 
Ap >K-1. 
Chebyshev’s inequality states 
Var(A;) 
é ; 


Pr[A, <= As -—e«l<s 
Since A = A, and A, = K — 1, we have 
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pike kKeiegee 
€ 


6.244 
<= “2gN ; (29) 
If we define 


C = set of all codes, 
C(S) = set of codes for which H(S|Z) s K-1- .«, 


and 
|A | = number of elements of A, 
then, since all codes are equally likely, (29) is equivalent to 
|C(S)| e 6.244 
Ley eae 


Now the set of codes for which H(S|Z) s K —1-—- forsome “ is 
UsC(S), and 





[Us C(Y)| = L 1C()|. 


S 
Therefore, 


IWC). y 1) 
|C| ey |Cl 

 . 6.244 

= Gt Y's) “aon: 


The number of possible “’s is the number of subsets of the set 


{1, --- , N} of size N — K, given by eal Pe Therefore, 





We CHI (Ny 8244 
|C| ONIN I gee 


If R = K/N, then 


( N < 1 QNnR) 
N-— K/~ J2xR(i — R)N 
where h(-) is the binary entropy function and 
lUeC(S)| 2 6.244 gN(H(R)=1) 
|C| ~ &V2rR(1 — RIN 


As long as the left-hand side of (30) is strictly less than 1, at least 
one code on the ensemble falls outside Uy C(S ), that is, provides 


(30) 


2152 TECHNICAL JOURNAL, DECEMBER 1984 


A>K-1-e forall “ This is guaranteed as long as 
6.244 gan) 2.49 DAO <D 
~ VaR. —R) VN VRQ-—R) VN 


Therefore, the existence of codes for which 


v2.49 gph" 
VR(1 — R)N 


2 


A=kK-1- 


is guaranteed. 
For R = 1/2, the worst case, this becomes 


2.23 


A=K-1-[, 
VN 


which is Theorem 4. 
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APPENDIX 
Statistics of Codes Used in Section V 


Here we present the expectations necessary to obtain the variance 
of A,, which is defined in Section V. 

Let n(1, 1) and n(2, 1) [resp. n(1, 2) and n(2, 2)] be the numbers of 
code words in the bin assigned to message 1 (resp. 2), which are 
consistent with distinct sequences z, and Ze. Recall that we have 
previously defined: 


AsLQ*Y 
BAQ* 
C A QN-K, 


Under the assumption that all partitions of the 2% code word 
sequences into 2% message bins are chosen with equal likelihood, then 
the probability of the four-tuple [n(1, 1), n(2, 1), n(1, 2), n(2, 2)] is 
the number of codes with those values divided by the total number of 
codes. 

This last ratio can be reduced to the ratio between (a) a numerator 
consisting of the number of ways that the bins (each of size C) 
corresponding to messages one and two can be assembled to include 
exactly n(1, 1) and n(1, 2) code words consistent with z, (from a 
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candidate pool of B elements) and exactly n(2, 1) and n(2, 2) code 
words consistent with z2 (from a disjoint pool of B elements); and (b) 
a denominator consisting of the total number of ways the two bins 
can be assembled. 

The numerator is 


v= [nc mc, 2) [te 2 m0 2) 
n(1, 1), n(1, 2)} | n(2, 1), n(2, 2) 


A — 2B 
‘1C — n(1, 1) — n(2, 1), C — n(1, 2) — n(2, 2))’ 


which is the product of the number of ways the elements consistent 
with z, can be drawn, times the number of ways the elements consist- 
ent with z. can be drawn, times the number of ways the elements 
consistent with neither can be drawn. 


The denominator is just 
A 
d= (cc): 


i.e., the number of ways that two arbitrary sets of size C can be drawn. 
The notation we have used is the standard trinomial coefficient: 


XV 4 Xx! 
Y,Z]) ~ yiZux—-Y-Z)!’ 


defined to be zero when any of Y, Z or Y + Z are greater than X, or 
when any of X, Y, or Z are negative. Defining, for the sake of notational 
compactness, 


a A n(1, 1) 
6 4 n(1, 2) 
vy 4 n(2, 1) 
6 4 n(2, 2), 
then 
B B A — 2B 
a, B 7, 6 C= a4, C= 6-9 
Pla, B, Y 5) S43 7 a 
A 
(che) 


The quantities needed for the analysis of Section V are a, 
a’, a’ B?, ay’, and a’y”. The actual calculations are involved and 
unenlightening. Everything can be evaluated using the relationships 


let )-C29 
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and 
xX ~ X-1 
€ ,) y=x(% 1, 7) 
If we define the auxiliary quantities: 


_ (B= i)(C-j) 


T; A-j 


and note since BC = A, that Ty = 1 and Tj: < T; so long as j is less 
than the smaller of B and C, then 

















Pad eT 
and 
at =l1+ TT; + 67, T> + T, T,T3. 

Also, 

= _A-C XC — 1)(B -2) (C —1)?(B —- 2)(B — 3) 
op = 4a ls A-2 | (A-2)(A—3) i 
x3 _A-B 1 4 B= DC = 2) J aie ee 
oes aan | A-2 (A-2)(A—3) |’ 
and 

x3 A 2(.B-1)(C-1) (B-1)?(C- 1) 
od se eS PY ; 


Referring back to the quantities needed in (25), we have 
M, = af — (a7)? 

=1+ TT; + 6T;, T2 + T,T2T3 = (1 + Ty" 

= 5T; + 67, T> + T,T2T3 — T:. 
Since T3 < T, < T;, then 

M, < 57, + 57? + T3. 

Next 

Mz = (C — 1)[n*(1, 1)n*(2, 1) — (n*)’] 

= (C — 1)[a’y? — (a”)"] 
A-B 2(B — 1)(C — 2) 

ony A=8fi 4 A= 


A-l 
(B -— 1)?(C — 2)(C — 3) 
=a J-a+ny}, 
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Now 
1 (B-1)(C — 2) _ C= 2A > 1. AC = A- C+2-A_, 
Ti, A-2 “CalLA=o' ACZA= C4020 : 


since A > C. Similarly, [(B — 1)(C — 3)]/(A — 3) < T;, so 





A-B me : 
<c-9[4=2asan+ry a+ ny 


=(C-1) c + 7) (4-2-1) 


__ (B-1C-1) 
= eat (1 + T;) 


= —-T,(1 + Tay 


Since a’? is just a6? with the roles of B and C reversed, Mz and 
Mz are “dual” to each other in this sense. Therefore, M3 satisfies the 
same bound as M2: 





M; <= -T\(1 + T,)?. 
The last term needed is 
= (B — 1)(C — 1)[n*(1, 1)n?(2, 2) — (n’)?] 
A 2(B — 1)(C — 1) 


-(e-ye-1 {4 |i + Aad 


(Ba De a4)? 7 
t ere aed) 


2 


A’=-1 A-2 
A(A — 1) 


(A — 2)(A — 3) 
= (B-1)(C- 1) + (eis 


of A(A—-1) 
on (a — (A — 3) | 


=e yo- | : ea) Z Aeron) 





= (B= Ie = 1) 


Ti a+ Ty] 





A- (A — 2)(A — 3) 
2 ee 
= 7,+4T; A—2 
» 44-6 Sins 
oT paca eo 


All nontrivial cases of interest have K and N — K = 2, so that B and 
C are each =4, which implies that 


2156 TECHNICAL JOURNAL, DECEMBER 1984 


(B-1)(C-—1)=BC-B-C+1sA-7, 











so that 
(B au: Desa 
Therefore, 
M, = 7, +47, 44 38 
<T; +4 - Oo, + ra 


Since T, < 1, then T? < T,, so 





m.<on +7 (428 za 


A-2 A-2 
4A — 26 
= 57 Qype fe 
57, + Ti A—2 
<5T, + 47%. 
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Stationary and Cyclostationary Finite Buffer 
Behaviour Computation via Levinson’s Method 


By M. H. ACKROYD* 


(Manuscript received November 8, 1983) 


A model for a finite buffer is described. The equilibrium equations for the 
number in the buffer have Toeplitz form. This enables Levinson’s method, 
which is widely used in the computation of optimal linear filters, to be used 
to compute efficiently the distribution of the number in the buffer, for both 
stationary and cyclostationary systems. The method can also be used to 
compute distributions in other, mathematically similar, queueing problems. 
These include the computation of the waiting time distribution in discrete 
time G/G/1 queues. 


Il. INTRODUCTION 


Buffer analysis is important in the design of data communication 
systems in which messages are queued whilst awaiting transmission. 
In particular, it is important to be able to choose the capacity of a 
buffer so that the probability of overflow is acceptably small. Methods 
for computing the distribution of the number in a finite buffer con- 
tinue, therefore, to interest telecommunications engineers. 

Various authors have dealt with computing the distribution of the 
number in a buffer (see, for example, Refs. 1 through 7). These 
references deal with models where the arrival statistics are constant, 
except for Ref. 7, which discusses the case where the distributions of 
successive arrivals at the buffer vary cyclically. This case arises when 
the arrivals are messages from cyclically polled sources, with differing 


* AT&T Bell Laboratories. 
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characteristics. In such cases, the equilibrium distribution of the 
number in the buffer varies cyclically, and the behaviour is termed 
cyclostationary. The present paper deals with both stationary and 
cyclostationary systems. 

Dor’ and Chu? studied systems where a single character is removed 
from the buffer at a time. In such cases the distribution of the number 
in the buffer satisfies a linear recurrence relation, from which the 
distribution can be computed directly. Chu® and Rudin‘ dealt with the 
distribution of the number in the buffer in systems where up to N 
characters can be transmitted at a time. Chu used Gaussian elimina- 
tion for the direct solution of the equilibrium equations for buffer sizes 
of up to 50 or so. Rudin avoided the direct solution of the equilibrium 
equations by dealing with the transient case. By starting with a known 
distribution, such as the one corresponding to an empty system, the 
successive distributions can be computed iteratively. The equilibrium 
distribution can be obtained by continuing the iterations until conver- 
gence, to within an appropriate criterion, is obtained. Bagchi and 
Templeton’ also applied this approach to the treatment of finite buffer 
problems. The method is applicable to stationary, to cyclostationary, 
and, indeed, to other nonstationary problems. Each iteration involves 
the computation of a discrete convolution. (Later in the paper the 
method is referred to as iterated convolutions.) For systems with a 
high utilization factor and large buffer size, the total volume of 
computation can be immense—even when the convolutions are com- 
puted efficiently via a fast Fourier transform algorithm. Except for 
this disadvantage, experience shows that the method is a satisfactory 
way of obtaining the equilibrium distribution. 

Leihs and Kobayashi® avoided the difficulties of exact computation 
by obtaining bounds on buffer overflow probabilities. They exploited 
the similarity between the fundamental equation for the number 
in the buffer and the fundamental equation for the waiting time in a 
G/G/1 queue. From known bounds on the G/G/1 waiting-time distri- 
bution, they derived bounds on the overflow probability in finite buffer 
systems. 

It seems that the solution of the equilibrium equations for buffer 
sizes such as one hundred to one thousand or more has widely been 
considered impractical. In this paper, it is shown that the matrix of 
the equilibrium equations, for one finite buffer model, has Toeplitz 
form, i.e., the elements on any given diagonal are all equal. This 
enables Levinson’s method,®*” which is widely used to solve Toeplitz 
equations in the design of optimal linear filters, to be used to compute, 
economically, the stationary distribution of the number in the buffer. 
To solve for K unknowns, Levinson’s method requires computer time 
proportional to K?, rather than K°, as needed by general methods of 
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solving linear equations. The paper also shows that, in the cyclosta- 
tionary case, the matrix of the equilibrium equations has block Toe- 
plitz form, i.e., when partitioned, the submatrices on any given diag- 
onal are all equal. This enables the extension of Levinson’s method to 
the block Toeplitz case, which is used in multichannel linear filter 
design,”° to be used to compute, economically, the cyclostationary 
distribution of the number in the buffer. 

One commonly used model for a finite buffer has equilibrium equa- 
tions for the distribution of the number in the buffer that do not have 
Toeplitz form. However, it is shown that the system can be represented 
as a cyclostationary one, with period two, so that it becomes possible 
to use Levinson’s method to solve its equilibrium equations efficiently. 
Finally, the paper points out that Levinson’s method can supply 
numerical solutions to other queueing problems whose form is math- 
ematically similar to the one dealt with here. These problems in- 
clude the calculation of the waiting time distribution in discrete time 
G/G/1 queues and cyclostationary queueing problems arising in the 
performance analysis of clocked schedules for real-time software. 


Il. FORMULATION—STATIONARY CASE 


Time is assumed to be divided into a sequence of intervals. The 
number of units in the buffer just prior to the end of each interval is 
the quantity of prime interest. The units may be bits, characters, 
messages, or other entities, depending on the application. During the 
nth interval, there may be both arrivals and departures. The net 
potential number of units arriving in the interval, a number that can 
be positive or negative, is denoted by X,. The number in the buffer is 
constrained to the range [0, K], so that the number in the buffer at 
the end of the nth interval is given by 


Yn = min[K, (Y,-1 + Xn)"], (1) 


where (z)* denotes max(0, z). 

There are various physical interpretations of this formulation. As 
one example, suppose that V,, units arrive during the nth interval and, 
at the end of the interval, as many units as permitted by a packet size 
W,, are removed from the buffer for transmission. X, in (1) then 
represents the difference V, — W,. With this interpretation, the 
number in the buffer during the interval can exceed K. If the limit on 
the number in the buffer were imposed throughout the interval, rather 
than just at its end, the system would be correctly described not by 
(1) but by 

Yn, = {min[(Yn-1 + Vn), K] — W,}*. (2) 


The behaviour of this system can differ somewhat from that of (1), 
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such as when a large value of V, is accompanied by a similar large 
value of W,,. The computation of the distribution of the number in 
the buffer for a system described by (2) is discussed in Section VI. 

Another interpretation of (1) is applicable when units arrive at the 
buffer in batches at purely random instants in continuous time and 
when, also at purely random instants, units are removed, in batches, 
for transmission. With this interpretation, X,, if positive, represents 
the number of units arriving in a group. If X,, is negative, it represents 
the number of units offered transmission at an instant. 

The successive X, are assumed to be independent and, in the 
stationary case, identically distributed, with distribution {a;}, i.e., 


a, = probability[X, = R]. 


The cumulative distribution of Y,, with the system in equilibrium, is 
{qa}, i.€., 
gz = probability[Y, < kl]. 


The cumulative distribution of Y,-; + X, is, because of the independ- 
ence of X, and Y,-1, the discrete convolution of {a,} and {q,}. Because 
of this and because Y,, is constrained to the range [0, K], the cumulative 
distribution of the number in the buffer can be expressed in terms of 
itself as 


0, k<0 
dh = Y gmarem Osk<K 
1, k>kK. 


This leads to the following system of equations for the K unknown 
points in the distribution {q,}: 


do d, s+ dK-1 do bo 
d_; doy -++ Ako qi by 
= bi (3) 
d«x-1) d-—K-2) +++ do qx-1 bx-1 
Dx 
where 
do =l]- ao 
d;, = —a-_p, k= +1, +2,---,+(K-—-1) 


b.= DY Gem, k=0,1,---,K-1. 


m=K-k 
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The equations in (3) could be solved by the use of a general-purpose 
routine for the solution of linear equations. However, for large K, the 
computer time and storage needed would be impractical. But the 
matrix in (3) is a Toeplitz matrix, i.e., one whose elements on any 
given diagonal are all equal. This enables Levinson’s method to be 
used for the economical solution for the unknown probabilities. 


Ill. LEVINSON’S METHOD 


When used to solve a system of Toeplitz equations with K unknowns, 
Levinson’s method requires computer time proportional to K?, rather 
than the K? required by general methods for the solution of linear 
equations. In the process of solving (3) for a buffer size K, it also yields 
the distributions for buffer sizes 2, 3, --- , K — 1. This is an advantage 
when the purpose of the computation is to determine a suitable buffer 
size for a given application. 

In signal processing applications, the elements of the matrix are 
normally autocorrelation coefficients, for which d, = d-,, a condition 
that normally does not apply in the buffer problem. Computer pro- 
grams based on Levinson’s method for signal processing applications 
are normally written to exploit the symmetry of the autocorrelation 
coefficients. Such programs, therefore, require modification before 
they can be used for the finite buffer problem. 

Levinson’s method, in a form suitable for the stationary finite buffer 
problem, is outlined as follows. At the pth stage of the recursion, the 
solution vector* q(p) = [qo(p), qi(P), «+» Yp-1(p)]” has previously 
been obtained, together with two auxiliary solution vectors, x(p) and 
y(p), which satisfy 


Xo(p), x1(p), ea Xp-1(P) D = ux(p), 0, ae 3g 0, 0 (4) 
Yo-1(P)s Yo-2(P), «+>, ¥o(P)] 0,0, +++, 0, u(p) | 


where xo(p) = yo(p) = 1. The pth stage of the recursion involves two 
steps: 

Step 1. Compute x(p + 1), y(p + 1) from x(p), y(p). 

Step 2. Compute q(p + 1) from q(p) and y(p + 1). 

x(p + 1) and y(p + 1) are formed as linear combinations of x(p) 
and y(p): 

[xo(p + 1), x1(p + 1), +++, Xp-1(p + 1), xp(p + 1)] 
= [xo(p), x1(p), +++» Xp-1(p), 0] 


- Cx(p a 1)[0, Yp-1(P), Ng yi(p), yo(p)] (5a) 


*The notation is altered here to denote explicitly the buffer size p to which the 
solution corresponds. 
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[yp(p + 1), yp-a(p + 1), ---,¥1(p + 1), yo(p + 1)] 
- Cy(p + 1)[xo(p), x1(p), sala Xp-1(P), 0] 
+ [0, Yp-1 (Pp); ear | yi(p); yo(p)]. (5b) 


The coefficients c,(p + 1) and c,(p + 1) are chosen so that the 
computed x(p + 1) and y(p + 1) will satisfy an equation like (4). For 
this, 


c,(p + 1) = —v,(p)/uy(p), —¢y(p + 1) = —vy(p)/uz(p), (6) 


where 


U;(p) = ¥ Xn(P)dz—p 
k=0 


p-1 
Vy(p) = » yr(p)dp—k (7) 
and where 


—1 
usp) = Y aa()dy = te(D — 1) + pa Pdp- 


= 
u(p) = E ye(pd-a = wy(D— 1) +Ips(Pldy 8) 


The first step in the pth stage of the recursion consists of evaluating, 
in appropriate order, (5) through (8), so that x(p + 1) and y(p + 1) 
are obtained. 

The solution vector q(p + 1) is obtained as a linear combination of 
q(p) and y(p + 1): 


[go(p + 1), +--+, Qp-i(p + 1), gp(p + 1)] = [Go(p), ---, Gp-1(p), 0] 
+ e,(p + 1)[yp(p + 1), ---, yi(p + 1), yo(p + 1)J, (9) 


in which the coefficient c,(p + 1) is chosen so that q(p + 1) is a 
correct solution. For this, 


Cq(p +1)= [bp am Y(p)]/Up(p +1), , (10) 
where 
vip) =D av(P)di-p (11) 


The evaluation of (9) through (11), in proper order, completes the pth 
stage of the recursions. The recursions are started with xo(1) = yo(1) 
= 1, vx(1) = Vy (1) = do and go(1) = bo/do. 

These formulas can be programmed simply; a set of Fortran 
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subroutines to compute the stationary finite buffer solution via Lev- 
inson’s method used a total of 49 lines of code. 


IV. NUMERICAL EXPERIENCE 


Accumulation of computational error is a possibility with recursive 
computations. However, the Levinson method is known to be numer- 
ically stable,’! and practical experience shows that it can give adequate 
accuracy for buffer analysis. 

The accuracy of results obtained by the use of Levinson’s method 
has been checked by comparison with results obtained by iterated 
convolutions and with known analytical solutions. For example, when 
a, = 1— a, a_; = a, a = 0 otherwise, the distribution of the number 
in the buffer is a truncated geometric distribution, with parameter 
(1 — a)/a. The computations were performed in double precision 
arithmetic on an IBM computer, in which numbers are represented to 
about 17 decimal digits and where chopping arithmetic (rather than 
rounding) is used. The combination of large buffer size and (1 — a)/a 
close to unity is least conducive to accuracy. However, even with a 
buffer size of 1900 and (1 — a)/a = 0.99, results accurate to seven and 
eight decimal places were obtained. A CPU time of 26 seconds on an 
IBM 3081/K was needed to produce the solution for a buffer size of 
one thousand. 

By way of contrast, 126 seconds of execution time was required for 
a 5K VIC-20* computer, programmed in BASIC, to give the solution 
for a buffer size of 48. This was the largest size permitted by the small 
memory of this small machine. The VIC-20 computer gave an accuracy 
of eight decimal places on the various examples that were tried. 
Levinson’s method thus makes practical the solution of the equilibrium 
equations, for moderate buffer sizes, even on small personal computers. 


V. THE CYCLOSTATIONARY CASE 


The previous sections discussed stationary systems in which the 
distributions of X, and Y, are independent of n. However, in some 
situations the distribution of X,, varies cyclically with n. An example 
is where the arrivals come from a set of data sources, having differing 
characteristics, which are polled cyclically. This section discusses 
systems described by the same fundamental equation as before, (1), 
but where the distribution of X,, varies cyclically. This distribution is 
denoted by 


Qkn = probability|X, = k]. 


* Registered trademark of Commodore Electronics Ltd. 
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With the system in equilibrium, the distribution of the number in the 
buffer will also vary cyclically. The cumulative distribution of the 
number in the buffer at the end of the nth interval is denoted by 


den = probability[Y, s k]. 


Because the distributions vary cyclically, it is sufficient to consider 
just one period, e.g.,.n =0,1,---,N-1. 

With the system in cyclic equilibrium, the distribution of the number 
in the buffer at the end of the (n + 1)th interval can be expressed in 
terms of the distribution at the end of the nth, n being computed 
modulo N. 


0, k<0 
dknti = > QmnQk-m,n+1) O<xk<K 
1, k= K. 


This leads to a set of equations, for the KN unknown probabili- 
ties, whose matrix has block Toeplitz form; i.e., when partitioned into 
N XN submatrices, the submatrices on any given diagonal are all 
equal. The equations can be written 


do di see Ake do bo 
d_, do --»  Ak-2 qi b; 
d-(x-1) G(K-2) --- do Qk-1 bx-1 


where the submatrices and subvectors are given by 


dk,0 > a_m,o 
m=K-—k 
feo] 
qk,1 Y Q—m,1 
m=K-—k 
Gr = b; = 
co 
Qk,N-1 yy GmN-1 
m=K—k 
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Ok 0 --» QO —A_-ko 


—a_R1 Or --- 0 0 
d; a ’ 
0 QO «++ && 0 
0 O +++ —a-pn-1 dp 
where 
q5 k = 0 
Op = 

0, k#0. 


Block Toeplitz equations occur in the formulation of multichannel 
optimal linear filters. Levinson’s method was extended to the block 
Toeplitz case for computing the coefficients in such filters.?!° For 
block Toeplitz systems the method follows the outline given in Section 
III, but with operations on scalar elements replaced by the correspond- 
ing operations on subvectors and submatrices and with the matrices 
corresponding to the c’s in (6) computed via the solution of linear 
equations. Programs for computing the coefficients of optimal multi- 
channel linear filters require modification before they can be used for 
the cyclostationary buffer problem. This is because they are normally 
written to exploit the symmetry d, = d”, that exists in the multichan- 
nel filtering problem but not, generally, in the cyclostationary buffer 
problem. 


VI. CYCLOSTATIONARY REPRESENTATION OF STATIONARY QUEUES 


The description of a finite buffer by (2) corresponds more closely to 
reality, in many cases, than its description by (1). As mentioned before, 
with (2), the number waiting never exceeds the buffer size. However, 
the equilibrium equations for the distribution of Y/ in (2) do not have 
Toeplitz form, and so Levinson’s method cannot be applied directly 
to their solution. However, Levinson’s method can be applied when it 
is recognized that a stationary system described by (2) can be repre- 
sented by a cyclostationary system, described by (1), having a period 
of two. 

Xn, the net arrival size in (1), is identified, alternately, with the 
arrival size and the service size in (2). That is, 


Vie n=2m 
Ay 
Wa n=2m+1. 


The distribution of X,, alternates between the stationary distributions 
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of V,, and W,,. The equilibrium distribution of Y, varies cyclically 
and, for even n, is identical to the stationary distribution of Y;/. 
Levinson’s method can thus be used to obtain the distribution Y;, by 
solving the block Toeplitz equations corresponding to (2) and retaining 
just the distribution for even-indexed Y,,. 

In most cases, the behaviour of a system described by (1) is very 
similar to one described by (2). They differ appreciably only in cases 
where there is a significant probability of buffer overflow due to a 
large value of V,,, which is accompanied by a similar value of W,,. As 
an example of a case where the behaviour of the systems differs 
appreciably, results were computed for a buffer size of five where V, 
and W,, are both geometrically distributed, with parameter 0.5. The 
computed distributions are as follows: 


dr, system described by Gr, system described by 


R (1) (2) 

0 0.2500 0.1429 
1 0.3750 0.2857 
2 0.5000 0.4286 
3 0.6250 0.5714 
4 0.7500 0.7143 
5 | 1 


VII. OTHER APPLICATIONS 


It is often necessary in telecommunication system analysis to cal- 
culate the distribution of the waiting time in a queueing system. A 
stationary G/G/1 queue—i.e., where the distributions of interarrival 
times and service times are arbitrary—often forms a suitable model of 
the system to be analyzed. The fundamental equation for T,, the 
waiting time of the nth customer in a G/G/1 queue, is 


Th = (T,-1 + U, 1)", (12) 


where U,-, is the difference between the service time for the (n — 1)th 
customer and the interarrival time between the nth and the (n — 1)th 
customer. When K in (1) is sufficiently large that the probability of 
Yn-1 + X, exceeding K is negligible, (1) has, effectively, the same form 
as (2). A method for computing the distribution of the number in a 
system described by (1) can, therefore, also be used to compute the 
distribution of waiting times in a G/G/1 queue with discrete service 
and interarrival times. Some error is incurred by the use of finite K 
but, by choosing K sufficiently large, this error can be made small 
compared with arithmetic error. 

Many switching systems use a processor whose time is shared among 
different tasks. The tasks have various requirements in terms of the 
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delay that can be tolerated between the demand for the execution of 
a task and the completion of its execution. A commonly used scheme 
for controlling the execution of such tasks is a clocked schedule.” The 
tasks have different priorities, and the schedule specifies which com- 
bination of tasks is to be executed in each interval between clock 
instants. Fredericks’* has shown that the resulting multiclass priority 
queueing system is equivalent to a set of single input G/G/1 queueing 
systems. In each of these queues the distribution of the quantity 
corresponding to U, in (12) varies periodically. Fredericks approxi- 
mated the periodically varying queue by a stationary queue. This gives 
accurate results for long delays but gives no information about the 
short time variation of delays that results from the varying nature of 
the system. Because of the similarity between (1) and (12), Levinson’s 
method for solving block Toeplitz systems permits the computation 
of the cyclically varying distribution of the waiting time in a discrete 
G/G/1 queue in which the distribution of U,, varies periodically with 
n. The method thus provides a means for the analysis of short-term 
delays in clocked schedule systems, as well as longer delays. 


VII. CONCLUSION 


Levinson’s method provides a practical means for computing the 
equilibrium distribution of the number in a finite buffer, by explicit 
solution of the equilibrium equations. On a mainframe computer, the 
method is practical even with quite large buffer sizes. With moderate 
buffer sizes, it becomes feasible to do the computations even on a 
small personal computer. The method also provides a practical means 
for computing the waiting-time distribution in discrete time G/G/1 
queues. 
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Distortion Analysis on Measuring the Impulse 
Response of a System Using a Crosscorrelation 
Method 
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In this paper we analyze the characteristics of a crosscorrelation method 
for measuring the impulse response of an unknown system. This analysis may 
also be of some interest in the area of synchronization of pseudonoise systems. 
In the first part of the paper a closed-form expression for the main component 
of the probing pulse is derived. This result may be used to quickly estimate 
the effect of various system parameters on the measurement’s accuracy. In 
the second part results are presented for two system configurations. 


I. INTRODUCTION 


Among the various methods for measuring the impulse response of 
a linear time-invariant system, a simple and effective method is to 
excite the system with a white noise-like input and crosscorrelate the 
output with the input. Actually, instead of a random white noise, a 
pseudonoise waveform may be used as an input.’ Such measurement 
systems are designed to produce the same effect as exciting the 
unknown system with a probing pulse having a flat spectrum over the 
entire frequency band of the unknown system. 

For these measurement systems the actual implementation is of 
fundamental importance, and various schemes have been proposed 
depending upon the system to be measured; among others we recall 
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Refs. 2 and 3 (see also the extensive bibliography in Ref. 1). This 
paper presents an analysis of an implementation introduced by D. 
Cox,” which is particularly attractive in radio environments when a 
high-resolution (i.e., wide bandwidth) probing pulse is required. The 
main advantage of this approach is in the hardware implementation, 
since the measurement system requires very little control logic (see 
Ref. 2). 

Information about the effect of actual implementations on the 
accuracy of this measurement technique is scarce. With this analysis 
simple formulas are derived that display the practical effects of the 
various system parameters, thus offering new tools to a system de- 
signer. 

Section II of this paper reviews some basic concepts on measuring 
the impulse response of a system by using crosscorrelation methods, 
and illustrates Cox’s methods from a heuristic point of view. This is 
analyzed in detail in Section III, where a closed form for the probing 
pulse is derived. In Section IV the analysis is particularized to different 
system configurations. Incidentally, we note that this analysis may 
also be of some interest in the area of synchronization of pseudonoise 
systems.° 


I. PRELIMINARIES 


The pseudonoise waveform that is used to excite the system is 
usually obtained by amplitude modulating a pulse g(t) with a pseu- 
dorandom sequence {a,}, n = 0,1, --- , M — 1; ane{—1, 1} so that the 
autocorrelation sequence 


1 M-1 
Ca(k) A uM > Q;Q(i-k)modM (1) 
has the form 
1, for (k)mod M = 0 
clk) =4 (2) 
—_— therwise. 
M re) 


For simplicity of implementation we consider the case where {a,} is 
generated by using a shift register with feedback.* A block diagram of 
the entire system to measure the impulse response of an unknown 
system with impulse response a(t) is illustrated in Fig. 1. If the output 
of the shift register with clock frequency f, = 1/to is used to amplitude 
modulate a pulse g(t), the output of the pseudorandom sequence 
generator r(t) is just the repetition with period T' = Mt of the 
pseudorandom waveform s(t) given by 
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| fo = Mto 






















PSEUDORANDOM UNKNOWN FILTER y(t} 
SEQUENCE SYSTEM A(t) 
GENERATOR Q(t) 


DELAY 7 


PSEUDORANDOM 
SEQUENCE 
GENERATOR 


fo = I/to 


Fig. 1—Schematic diagram of the measuring system. 


M-1 
s(t) = ) aig(t — ito). (3) 
Thus 
r(e) = ¥ ag(e ~ it) (4) 


with the understanding that a; = djmoam. The autocorrelation function 
of r(t), 


h 
OQ aan i Rae tide 
2t; —ty 


t Tam acaed 


T 
= i r(n)r(m — t)dn, (5) 


is periodic with period T. If g(t) has finite support, say g(t) = 0 for 
t<0 and t> to, then c,(t) has a simple expression given by 


M 
c-(t) = )) ca(k)c,(t — Rto), for 0<t<T, (6) 
k=0 
where 
i.{* 
Cy(t) = ra { &(n)g(n — t)dn. (7) 
0 0 


In the particular case where the output of the shift register feeds 
directly the “unknown system,” we have that* 


* The function rect(-) is defined as 


1 for bag 
pie: {" 2 = 2 





a 
rect b = 
0, elsewhere. 
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t — to/2 


g(t) = rect ———_. (8) 

to 

Then 
Pe ee. | xz 

c,(t) = ( il rect Dh (9) 

and c,(t) has the well-known expression: 

__1i A\/, _lé t L 

c,(t) = Mu + ( + x) ( i! rect Oh for |t| < 3 (10) 


as shown in Fig. 2. The “temporal resolution” of this pulse is seen to 
be on the order of tf). Thus, to improve resolution, one might increase 
the shift register clock frequency. 

For some applications it might be easier to change the pulse shape 
g(-) rather than the shift register clock frequency; Fig. 3 shows c,(t) 
when g(t) = V2 rect[(t — to /4)/(to/2)]. 

Referring again to Fig. 1, the output of the “unknown system” is 
multiplied by a time-shifted version of the input r(t — 7) to give p(t). 
This signal is then filtered with a low-pass filter having impulse 
response h(t). If h(t) is an ideal integrator between 0 and T, i.e., 

T 
t — 


1 2 
h(t) = T rect TF ; (11) 





Fig. 2—Autocorrelation function for g(t) = rect[(t — to/2)/(to)]. 
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Fig. 3—Autocorrelation function for g(t) = V2 rect[(t — to/4)/(to/2)). 


then 


Je ole 


yr & y(t) 


{ p(n)dn 
t-T 


ff [a(Or(n — Odgl[r(n — 7)]dn 


T 


- { a(Sje-(r — sdf, (12) 


i.e., the output has a constant value y, equal to the convolution of 
a(-) with c,(-) evaluated at r. 

Assuming f, is larger than the highest frequency component of 
a(-), and M is large enough, we have that y, is approximately propor- 
tional to a(r).? From (12), c,(-) will be referred to as the probing pulse 
in the measurement system. A straightforward implementation of the 
system in Fig. 1 has been presented in Ref. 3. 

An alternative configuration that has the benefit of a simple imple- 
mentation has been proposed by Cox.” The baseband equivalent of 
Cox’s carrier system scheme is shown in Fig. 4. The only difference 
with respect to the system of Fig. 1 is that the output of the “unknown 
system” is multiplied by a pseudorandom waveform that is identical 
both in character and in timing with the transmitter sequence, but it 
is produced by a different clock frequency f;. As in Ref. 2 f2 is related 
to f. by 
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fie = Wto 
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Fig. 4—Schematic diagram of the measuring system using Cox’s method. 


foe ( - z) (13) 


and K will be a parameter of our system. 
In general, consider the function 


, A — 
Cr r(7) — sie 
where 7 is the delay at time t = 0 between the two sequences. As time 
elapses the delay decreases by the amount (t/t$)(to — to) = t/K, so 
that 


T 
i r’(n)r(n — r)dn, (14) 


1 : aes fy 
7 ile r’(n)r(n — 7)dn = cy, (: - 1, for t2T. (15) 


In (15) the approximation is due to the fact that the integral depends 
on the symbols {a,} under integration. Now, if K is large enough we 
can assume that 


crr(t) = ¢,(t). (16) 


In this case, for an integrate and dump filter, we have that 


ne a & 
y(t) = f a Jevr (:- as dg 


sis = T 
2 f alder ( = tar (17) 


y(K + T) = f a(Sje-(t’ — §)df. (18) 





or 


Note that the right-hand side of (18) coincides with (12). Thus if K is 
large enough such that assumptions (15) and (16) hold, this measure- 
ment system is equivalent to the system of Fig. 1. Unfortunately, K 
cannot be set arbitrarily large, as this would give rise to two conflicts. 
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First, the duration of the experiment [see eq. (18)] would become very 
long, with the consequence that the assumption of a time-invariant 
system may no longer be true. Second, a very large K would imply a 
costly frequency synthesizer from which f; is derived.” 

In the next section we analyze the conditions under which assump- 
tions (15) and (16) may be considered valid. We consider also the 
effect of using a low-pass filter as h(t) rather than an ideal integrator 
as in Ref. 2. In general terms we shall see that the probing signal can 
be written as a sum of two terms: a useful term plus noise. A closed- 
form expression for the useful term is derived for a general filter 
impulse response h(t). Also, the dependence of the noise term upon 
the system parameters will be determined. 


Ill. PROBING PULSE DERIVATION 


To determine the actual probing pulse for the system of Fig. 4, we 
can consider the output function of the block diagram in Fig. 5. We 
shall determine the conditions on K and filter impulse response h(t) 
such that q(-) approximates c,(-). 

In Appendix A, a general expression for g(t) is derived for an 
arbitrary pulse shape g(t) (which for simplicity we assume to have 
finite support to). It is assumed that h(t) has finite support, i.e., 


h(t) = 0, for t<0, and t>t, (19) 
with 
th < K(M — 2)tp. (20) 


As we shall see, this assumption does not impose any serious limitation 
on the filter type for practical purposes. Typically, t, will be much less 
than K(M — 2)to, but the analysis applies as long as (20) is satisfied. 
Furthermore, we normalize h(t) such that it has unit area. Let 


h’(t) & Kh(tK) (21) 


and 


d(t) = f Cg(n)h’(n — t)dn. (22) 


r(t-T) 





r'(t) 


Fig. 5—Equivalent system to derive probing pulse. 
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d(t-7/2K) 





Fig. 6—The d(t) pulse for an integrate-and-dump filter. 


In Appendix A it is shown that g(t) can be written as the sum of two 
terms: 


q(t) = qo(t) + qn(t). (23) 
In particular, when g(t) is given by (8), go(t) has the form 


wt) =-E+(E+1)a(r-4), 


for -T+i<r-Z< to. (24) 
For a low-pass filter with t, < KT, if K >> 1 we have from (19), (21), 
and (22) that d(t) = c,(t). Comparing to (10) we see that go(t) coincides 
with c,[7 — (t/K)]. Moreover, if the bandwidth of the filter is small 
enough,* it is seen that the general term of the series (69) which 
defines q,,(t), W(m)-G(m/T) -A{(l/T) — (m)/(KT)], is very small, with 
the result that q,(t) is negligible if K >> M(G(f) and H(f) are the 
Fourier transforms of g(t) and h(t), respectively). Thus, if the band- 


* In practice, this condition is not in conflict with (19) for K, a large number; i.e., we 
require for a general filter shape T «< t, < KT. For special filter shapes t, may be as 
small as T; e.g., the ideal integrator discussed in the next section. 
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width of H(f) is small enough, and K is large enough, q(t) coincides 
with c,[r — (t/K)] as desired. A more quantitative analysis will be 
given in the next section for two particular impulse responses h(t). 


IV. EXAMPLES 


For g(t) given by (8), we shall evaluate qo(t) and q(t) for two extreme 
cases: first for an ideal integrate-and-dump filter, then for a single- 
pole Resistance-Capacitance (RC) filter. 


4.1 Integrate and dump filter 
Here h(t) is given by (11), thus 


t — T/(2K) 
T/K 
and for K > M we get 
K(M(,_M\_(t\ 
M\K 4K to) }’ 


M 
f 4 < to = 
or 0 | t| 05K 


h’(t) = i rect (25) 


M M 
i(e-Z)- for Me ii<t(1- 4) (26) 


for |t|2=t eae 
= lo OK} 


Plotted in Fig. 6 is d(-) for various values of M/K. We note that 
d(-) depends only on the ratio M/K and already for M/K < 0.5 the 
distortion introduced by h’(-) is very small. For a comparison with 
the ideal case we also present c,(t) as given by (9) (dashed curve in 
Fig. 6). 

With regard to the noisy component q,(t), according to the series 
(69) the main contributions to q,(t) are for 1 = +1. In Fig. 7 we plot 
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M 2M 


Fig. 7—Sequence profiles of noisy term qn(t). 


the envelopes of the sequences (1/to)|G(m/T)| and |A[(1/T) — 
(m)/(KT)]| in (69) for K/M = 6. Since from (52) 


ree me a 

|(m)| < (i+ H) (27) 
we can say that although q,,(t) depends on M, it can be made negligible 
by increasing K/M. 

Thus far, using general properties of a pseudonoise sequence, we 
have determined the exact shape of go(t) and got a rule of thumb of 
how to make q,(t) negligible. However, if an exact determination of 
gn(t), and therefore q(t), is desired, we need to know the sequence {a,}, 
n=0,1, ---, M-—1, explicitly. In the following an exact expression 
for q(t) is derived as a function of the system parameters. The output 
of the filter h(t) will be computed only for times corresponding to 
filter integration intervals that begin at integral multiples of the 
receiver sequence r’(-) subintervals tj. Since the width of q(t) is 
approximately 2Kt), the computed values will be sufficiently closely 
spaced to completely characterize the output. 

Here we consider only the case K > M; similar expressions hold for 
K < M. Let 


t — to/2 
g’(t) = rect peta (28) 
to 
[modulating pulse for r’(t)] and 
g”(t) = rect Oe ; (29) 
0 
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where 

té = T- (T’ - th) 
K-M 
K-10 
Thus g”(t) is the pulse that must be added to M — 1 g’(t) pulses to 


provide a total duration of T. Moreover, czg(t) and Czg(t) denote 
correlation functions, defined as 


(30) 


= to 


Cap(t) = te 


T i ‘ a(n)b(n — t)dn. (31) 


We introduce the periodic function z;(7) defined as 


T 
an) = f r'(Or(é — nd. (32) 


The subscript J indicates that the sequence {a,} in r’({) starts at ¢ = 
0 with the symbol a;. Substituting (4) for r(-) and similarly for r’(-) 
in (32), 27(y) can be written, for 0 <7 < T, as 


M [M-2 
2(Lip - HO = DY { YD ai-z+1 O41 Cgg (Jté — ito + 9 
5. \\gap 


+ Qy—141 OM-141 Cegr((M — 1)t6 — ito + o} ; 
L=1,2,---,M and 0<¢ fp. (33) 





Fig. 8—Probing pulse for an integrate-and-dump filter: M = 511, and K = 1000. 
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to t§ — to 
=, for |t|/ < 
T lel 2 
1 [t6 + to ts — to ti + to 
= i= — ltl), for —— < |t| < 
*( 2 | } 2 lel 2 
th + t 
0, for |t|= 7 (34) 
and 
to — to 
te to — té 
oe for |t| <~—2 
T [¢| 2 
1 [to + t? to — to to + to 
=) —{— - It for —— < |{t]| < ——— 
to + t 
0, for |t| ey (35) 


have finite support, we can limit the summation in j to just a few 
terms, namely, 


M (Jy 
2(Lt — ) = ¥ { DY Gitar Oj+1 Cyg (jt — ito + 9) 


i=0 Ldn 


+ aj—n+1 AM—141 Cgg”((M — 1)t§ — ito + of , (36) 


min (| — o_o | ,M- 2 (37) 
to Cc 


n (eed _M- 2). (38) 
to F 


[-]c and [-]r are the “ceiling” and “floor” functions, respectively. 
Finally, for t = T + Ito, J an integer, we have 


Fee? (: 4 7). (39) 
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where 


Im 





Ju 


Plots of q(t) shifted in time by —[(7T)/(2K)] [see eq. (26)] and scaled 
by the factor K are reported in Figs. 8 and 9 for M = 511 and two 
values of K. Specifically for x a multiple of tg /K we plot 


io efe (+3) 
=a(r+ 2-2), pag. (40) 


t6 


From these figures we can see that the noisy term is significant for 
M/K = 0.511. The maximum value of |q,(t)| in Fig. 8 is about —33 
dB. However, for M/K = 0.128 it is reduced to —43 dB (see Fig. 9). 


4.2 Single-pole RC filter 


Here, the filter impulse response is 


t 
(pd e® t20 
A(t) =< RC (41) 
0, t<0. 


Note that in order to guarantee condition (19), (AT)/(RC) must be 
much greater than 1, as will be the case. Let 





= . (42) 





Fig. 9—Probing pulse for an integrate-and-dump filter: M = 511, and K = 4000. 


IMPULSE RESPONSE MEASUREMENT 2183 


Then, from (22), d(t) has the form 


0, t > to 
1 t/to~1 
1——+B\e ® -1], 0<t<t 
to 
d(t) = ‘ tte tto-t (43) 
L+—+B\l—2e% +e ? ; —t<t<0 
‘0 
Hof 1 _l 
Be® \e® -2-—e ®]), t <— bo. 
The maximum value of d(t) occurs for 
_ 
t = tmx = —t8iln 2-e 8 
= —0.698to (for small 8) (44) 
and 
d(tmax) =1- B In 2 
= 1 — 0.698. (45) 





Fig. 10—The d(t) pulse for a single-pole RC filter. 
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However, for sufficiently small 8, d(-) is approximately symmetric 
with respect to the abscissa t = tg = —Gto. In Fig. 10 we report 
d(t — 0.96t,) for three values of 8. The dashed curve in Fig. 10 refers 
to c,(t). It is interesting to note that d(t) is independent of the 
sequence length M and is almost indistinguishable from c,(-) for 6 < 
0.2. 

The noisy term gq,,(t) can be treated similarly to the prior case. Here, 
the condition for negligible g,(t) is: 


RC 
ie 1. (46) 


For an exact evaluation of q(t) a similar procedure as above is used 
(details appear in Appendix B). Figures 11 and 12 show a shifted (by 
0.9 Bt)) and scaled (by K) version of g(-), g(-), for two values of RC 
and the same value of @. In Figs. 11 and 12 the maximum value of 
|qn(t)| is -34 dB and —44 dB, respectively. 


V. CONCLUSION 


Expressions for the probing pulse in a measurement system have 
been derived. The closed-form expression for the main component is 
particularly simple. This has allowed us to investigate the trade-off 
between the parameter K and the receiver filter complexity. In partic- 
ular, for the same system configuration as in Ref. 2, we can replace 
the ideal integrate-and-dump filter with a simple RC filter, but K must 





Fig. 11—Probing pulse for a single-pole RC filter: M = 511, RC/T = 0.978, and 6 = 
0.046. 
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Fig. 12—Probing pulse for a single-pole RC filter. M = 511, RC/T = 4.5, and 8 = 
0.046. 


be increased by more than an order of magnitude in order to maintain 
the same pulse shape and noise level. The larger K requires a more 
precise frequency synthesizer. The increased restrictions imposed by 
the RC filter relative to an ideal integrator may often be accommodated 
without modification of typical system performance capabilities.” 
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APPENDIX A 
Derivation of q(t) 


Since r(t) is a periodic function of period 7, we can consider its 
Fourier Series (FS) expansion: 
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r(t) = y Bie (47) 


If g(t) has finite support ft), the Fourier Coefficients (FCs) {#(n)} are 
given by 
Bn) = PL(n)F(n), (48) 
where ./(n) is the discrete Fourier Transform (FT) of {a,}, n = 0, 1, 
-+-M-1: 


L(n) -15 Tae (49) 


1=0 


and Y(n) is proportional to the FT of g(t), G(f), evaluated at f = 
n/T, 


Y(n) = . G (2) (50) 


For later use, two facts follow. First, o/(n) is related to the discrete 
FT of {ca(k)}, { Za(n)}, by 


B,(n) = | A(n)|? (51) 
and from (2) it is shown that 
a for (n) mod M=0 
G,(n) = ‘ i (52) 
Mu + 2’ otherwise. 


Second, we note that (1/M)| Y(n)|? is just the nth FC in the FS 
expansion of c,(t) [defined in (7)] in the interval [—(T/2), (T/2)]. 
Thus we can write 


T 
<3" (53) 


bia 1 2 jomat 
e(t)= % ~|P(n)\’e 7, for |t| < 
nao M 
Since r’(-) is related to r(-) by the equation 
r'(t) =r (: i), (54) 
to 


r’(t) is also a periodic function but of period T’ = Mto = [K/ 
(K — 1)]T, and its FS has the form: 


tines Bee eo (55) 


m=—o 
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Since K and K — 1 are mutually prime numbers, 
p(t) = r’(t)r(t — 7) (56) 


is a periodic function of period KT. Its FS has the form 


+00 pets 
pt) =» P(e "Kr, (57) 


=—00 


Upon substituting (47) and (55) into (56) we have 


t 


+00 +00 oe ee j2a—~(Kn+(K-1)m) 


pt)= Y Y A(n)A(mje Te *? (58) 
Comparison of (57) and (58) shows that #(l) can be written as 
A= YD Aina me. (59) 
Papa 


Making use of assumption (19) we can consider the FS of h(t) in 
the interval [0, KT]; its FCs will be given by 


&(n) = oy a(Z) (60) 


where H(f) is the FT of h(t). 
Finally, since q(t) is the convolution of h(t) with p(t), q(t) is a 
periodic function of period KT and we can write 


+00 


: I 
git)= Y Qe *, (61) 


[=—0o 
where coefficients { 2(l)} are given by 
QGB(l) = KT P(l)A(l). (62) 
Substituting (59) and (60) into (62), and this result into (61), we get 


+00 +00 “7 
2 
J T 


g(t) = KT 2 > B(n) B(m)e 
_ Kn+(K-1)m 


.O(Kn +(K- me" TF (63) 





Let / = n+ m; a more convenient expression for q(t) is: 
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q(t) = KT 2 x B(m)#&(l — m) 
Lom 


| gn. (h. 
Gh awne 6 C mm (64) 


In general, g(t) can be separated in a useful component qo(t), given by 
(64) for / = 0, and a noisy component q,(t), i.e., 


q(t) = qo(t) + qn(t), (65) 


where 


go(t) = KT y Rigen, (66) 


m=—0o 


A closed-form expression for qo(t) is now derived. Since #(—m) = 
&*(m), using (48), (51) and (52) we get 


ees 1 ox mat( +) 
qo(t)=— Ta | Y(mM)|?KW(—mM)e 7 \' ¥& 


Sea y To | Yim) |2KM (mer W (67) 
M oc, - VE ; 
Using general properties of FS, (67) becomes 
B. rat t..5 1 t 
g(t) =— M weedy d (: - RO is) + (i+ 7 d (: - £ 


for —T+t<7r-sSbt, (68) 


ml 


where d(-) is defined in (22). From (65) and (66) the noisy term q,,(t) 
has the form: 


abe ca 1 m\ 1 l—m 


l m pee ina(a-7) 
-H 7 = KT e e ; (69) 


where the primed summation excludes the | = 0 term. In the particular 
case when g(-) is given by (8), (68) simplifies into 


wi)=-t4(Es i)a(r-4) 


for —T + t<7T- <= lo. (70) 


Al 
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APPENDIX B 
Derivation of q(x) for a Single-Pole RC Filter 
From Fig. 5, and for h(t) given by (41), we have that 


t t-t 


at) = J aoe Bron — dk. (71) 


Let z;(n) [J has the same meaning as in (32)] be a periodic function of 
period T' defined as 


ar, _ar-t 
2(n) = f RC? RC r’(Or(F — n)d¥. (72) 
Let 
QT > RC (Q an integer), (73) 


so that the infinite range integral in (71) may be truncated to {t — 
QT, t}. As in the case of the ideal integrator we evaluate q(t) at discrete 
times: 





t= QT + Its (J an integer). (74) 
Since 
_,%_t—- QT 
I(to — to) = Iz Si agen (75) 
we have for the times given in (74): 
q(t) = 21 (: -* = . (76) 


The value of Q should be chosen as a compromise between accuracy 
in the results [see eq. (72)] and long computer execution time. For our 
purposes, a choice of Q = 8 was more than adequate. In order to 
simplify matters, we are also going to assume that 


K> Ma. (77) 


Under these conditions, r’(t) can be written as 


MQ-2 
r’'(t)= > ajarg’(t — jto) + ame-141 g’(t — (MQ — 1)to), 
j=0 


for 0<t<QT, (78) 


where 
Pi =te = (79) 
to 
and 
» ., K- MQ 
to =b 7 Te (80) 
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Also for 0 < 7 < T we have 
MQ 


r(t — ») = x Qj-r+r g(t — itp) + for 0<t—7ns<QT, (81) 
where L and ¢ are related to 7 by 
n = Lt — § L=1,2,.--,M, 0< F< bh. (82) 
Upon substitution of (81) and (78) into (72) we get 
MQ ( Ju 
z(Lt) — n) = Py By Oj-L+1 Aj+1 Cgg’ (ito — §, jto) 


+ Gj-141 AMQ-14+1 gg’ [ito — §&, (MQ — 1)to], (83) 


where 
: ito — £ 
Jm = min (a - 7 , MQ - | (84) 
to Cc 
Ju = min Poe , MQ - 2} (85) 
to F 


t to 
ea (a- 2, 0-4) 


ll 
{ P) 
8 + 
8 
a 
Q 
a t 
© 
al 
Gq 
—es 
oo} 
l 
Qa 
+ 
oO 
a 
oa 
Pe 
oe 
o 
+ 
Ist 
KS 
Q. 
oF 





to +t 
0, hag oo 
2 
ott 
b-a>--— : 
b-a+té/2 _tof2 2 
e RC _ @ RC iy 
b-a<--— : 
Z 
to — t 
_QT-a ° ‘ ad a aaerace : 
=p, Re 2 ahi 86 
ghG sg Re. : ea (86) 
2 
to — t 
ae 
to/2 b—a~t/2 Z 
ekC — @ RC 
t6 + to 
b-a<—— 
2 
tot+t 
0, ae er ae 
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i 1 QT-t t ti 
_ ci ARE t— bal w = 8 
be RC? x a+ 2) ( b+ Sa 





to + to 
0, page ees 
2 
to + to 
b—-a>-~—* 
9/2 _b-atto/2 2 
eFC —e RC ; t — tl 
page 
2 
tr, — t@ 
eae oe 
QT-b tg /2 tg /2 
-aT ae 87) 
ae ORE eRC —@ RC ia Bagh ( 
2 
t, — tf? 
b— bi Si 
_b-a-to/2 _ 6/2 2 
eS see Re, to + to 
bane 
2 
to + tg 
0, a aa ra 


Finally, g(x) as reported in Figs. 11 and 12 is defined only for x 
multiples of t/K as 


q|K(x + 0.98to)] 


z, (r+ P= Poe _ 5), nae (88) 
K to 


q(x) 
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Soft Decision Demodulation to Reduce the Effect 
of Transmission Errors in Logarithmic PCM 
Transmitted Over Rayleigh Fading Channels 


By W. C. WONG,* R. STEELE,* and C.-E. SUNDBERG* 
(Manuscript received May 16, 1984) 


In this paper we investigate soft decision demodulation applied to y-law 
Pulse Code Modulated (PCM)-encoded signals transmitted over Rayleigh 
fading channels by means of coherent phase-shift keying modulation. Each 
bit in the p-law PCM word is assigned its own soft decision demodulation 
erasure threshold. These thresholds are theoretically determined as a function 
of the input power level, channel s/n, and the relative mean square error 
power 6 that occurs because of the replacement by interpolation or prediction 
of those samples discarded in the soft decision demodulation process. We find 
that there is no advantage in applying soft demodulation to more than the 
first 4 bits of the 8-bit u-law PCM words, » = 255. When the input signal 
level was —17 dB (corresponding to peak overall speech s/n in the absence of 
transmission errors), the gains in overall speech s/n compared to basic y-law 
PCM and fixed weighted p-law PCM were 7 and 3.5 dB, respectively, when 
the channel s/n was 30 dB. More significantly, when the input signal level 
was reduced to —40 dB, the corresponding gains in overall speech s/n were 18 
and 12 dB. Simulations were performed using four concatenated speech 
sentences and a fading channel that was obtained from a mobile radio Rayleigh 
fading hardware simulator. The simulations were in reasonable agreement 
with our theoretical results. 


I. INTRODUCTION 
Digital transmission of logarithmically companded Pulse Code Mod- 
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ulation (log-PCM) over Rayleigh fading channels has been studied for 
Coherent Phase-Shift Keying (CPSK) and Noncoherent Frequency- 
Shift Keying (NCFSK).! The effect of transmission errors on the 
recovered analog signal can be most serious, particularly when the 
more significant magnitude bits are corrupted. To mitigate these 
effects numerous techniques can be employed, such as: channel pro- 
tection coding,” bit scrambling prior to transmission in order to combat 
burst errors,’ diversity schemes,** weighted PCM,°*” soft decision 
demodulation,*° and so forth. 

In this discourse we will focus on soft decision demodulation, a 
method that is implemented only at the receiver. As a preamble we 
need to distinguish between soft and hard decision demodulation. Let 
us commence with the observation that although the carrier is modu- 
lated by a binary log-PCM signal for its transmission over a Rayleigh 
fading channel, the demodulation process yields an analog rather than 
a digital signal. This loss of binary status is due to the channel 
imperfections and the noise in the front end of the receiver. The role 
of the regenerator may be viewed as an Analog-to-Digital Converter 
(ADC), identifying the binary signal that lies latent in the demodulated 
radio signal. When hard decision demodulation is employed, the output 
of the ADC is a logical one or a logical zero, the state being dependent 
on which side of the decision boundary the input analog level resides. 
By contrast, soft decision demodulation has more than two output 
states, and the occupancy of any state is associated with a reliability 
number, i.e., a description of our confidence as to whether the trans- 
mitted bit was a logical one or a logical zero. For the simple soft 
decision demodulator we have three states corresponding to three 
zones. The middle zone extends from —Z to +Z, and is known as an 
erasure zone. As the transmitted binary signal represents logical one 
and logical zero states by equal magnitude positive and negative 
voltages, respectively, a signal level that resides within the erasure 
zone at a sampling instant is deemed unreliable. This unreliable status 
is assigned because the level of the signal is not sufficiently positive 
or negative to warrant our confidence of assigning it as a logical one 
or logical zero. The other two zones stretch for input signal levels that 
are >Z, and <—Z, and signal levels at a sampling instant that are in 
these zones are regenerated with relative confidence as bits of logical 
one or logical zero, respectively. 

Various strategies can be employed when the analog input falls into 
the erasure zone. Typically,®° if one or more bits in the first M Most 
Significant Bit(s) (MSB) of an N-bit PCM word are deemed to be 
“unreliable”, the entire word is rejected. The resulting missing decoded 
sample is replaced by means of interpolation or prediction from 
neighboring decoded samples. The last (N — M) bits of the PCM word 
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are not subjected to soft decision demodulation because an error in 
these bits is decoded into a sample that is often more acceptable than 
a sample that has been imperfectly introduced by interpolation or 
prediction. 

It is our intention here to extend the work on soft decision demod- 
ulation for the Gaussian channel®"° to the case of the Rayleigh fading 
channel. We commence in Section II with a discussion concerning the 
digital noise power in log-PCM where the transmission is over a 
Gaussian channel and soft decision demodulation is employed. The 
problems associated with a Rayleigh fading channel are encountered 
in Section III. Specifically, the conventional fixed-width erasure zone 
is examined in Section 3.1, and in Section 3.2 we optimize a soft 
decision demodulation system where each bit is assigned its unique 
erasure zone. Section IV states our performance criterion, while Sec- 
tion V is concerned with presenting the theoretical performance of 
our 8-bit u-law PCM that utilizes an individual bit threshold soft 
decision demodulation system. Computer simulation results for u-law 
PCM speech transmitted over a mobile radio channel and utilizing 
soft decision demodulation are provided in Section VI. The final 
section summarizes our findings. 


Il. BASIC CONCEPTS 


When the log-PCM binary output signal is transmitted in the form 
of antipodal signals over an additive white Gaussian noise channel, 
the digital noise power produced when a soft decision demodulator is 
employed can be shown to be® 


M N 
@&=2Pp XA, + P > A, + PpiS?, (1) 
l=1 l=M+1 

where we have assumed that the bit errors occur independently. The 
first term in this equation is the digital noise power due to the presence 
of only one erroneously regenerated bit in one of the first M bits of 
the N-bit log-PCM words. These bit errors occur because the received 
signal amplitude resides outside the erasure zone at the instant of bit 
regeneration. Consequently, the erroneously regenerated bits are not 
designated as unreliable. The probability of an undetected bit error is 
Pp, while A;is called the /th single-error A-factor. For a full description 
of the single-error A-factors the reader is advised to consult Refs. 1, 
2,5, and 7. Suffice to state here that the definition of A; is 


A, & Ej{(xi — xi)*}, . (2) 


where E;(-) is the expectation of (-), and x; or x;; is the recovered 
speech sample in the absence or presence of transmission errors, 
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Respectively, i= 0, 1, , 2-1. The expectation is made over all 
i = 2% quantized level: “Note that for input speech samples x, the 
encoder noise power is given by 


ef + 2 = E{(x — x;)7}, (3) 


where ¢7 and ¢? are the quantization and clipping noise power contrib- 
uted by the encoder, respectively, and the expectation is formed over 
the source statistics. By contrast, A; is the average noise power in the 
decoded output sequence due to the existence of a single error in the 
Ith bit of all the transmitted N-bit log-PCM words. Observe that A, 
Ao, --:, Ayn are the first, second, ---, Nth, single-error A-factors, 
respectively, and relate to a single transmission error in the first, 
second, --- , Nth bits of the log-PCM words, respectively. We assume 
that the regenerated errors are statistically independent. 

The contribution to the total digital noise power due to an error 
occurring in the last (N — M) bits of the log-PCM words is represented 
by the second term in eq. (1). These least significant (N — M) bits are 
not subjected to soft decision demodulation, i.e., they are detected by 
hard decision demodulation. The average bit error probability for the 
hard decision demodulation is P. 

When one or more bits in the first M bits of a log-PCM word are 
deemed unreliable, the complete word is rejected. The probability of 
this event is Pr. The noise power due to replacing the original log- 
PCM words by those determined by interpolation or prediction is 
represented by the final term in eq. (1). Observe that this noise power 
is the product of the input signal power S? and the relative mean 
square error 6 of the correction process, namely, 

S? = E{(x — £)*}, (4) 
multiplied by its probability of occurrence, Pr. The value of 6 is 
determined by the quality of the correction process, having a low value 
when the corrections are accurate, and vice versa. For the type of 
correctors that are relatively easy to implement the value of 6 spans 
the range from 0.4 to 0.01. In eq. (4), x is the input sample, and < is 
the sample at the output of the receiver after the interpolation or 
prediction processes have been applied. We also note that 


Pr=1-(1—P,)” = MP,, (5) 


where P, is the probability that a matched filter output voltage is 
within the erasure zone +Z, where 


Z=TVE (6) 


and where T' is the soft decision erasure threshold and E is the bit 
energy. When Coherent Phase-Shift Keying (CPSK) modulation is 
used, the probabilities P, Pp, and P, are® 
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P 


o( Vi : 


E 
Po= Q( 2Zu+n) (8) 


E E 
P= @( 2Zu-m)-a( V2Za+n), (9) 


where E/N, is the channel s/n, N, is the one-sided spectral density 
function for white Gaussian noise, and 


Be —u?/2 
Q(x) Ton errrdu. (10) 


Threshold optimizations for the Gaussian channel are described in 
Ref. 8. 


Hl. RAYLEIGH FADING CHANNELS 


When the bits constituting the log-PCM signal are scrambled prior 
to their transmission over a Rayleigh fading channel, the burst errors 
that would have frequently occurred in the regeneration process at the 
receiver appear as random errors. Because of the scrambling process 
we assume that the bit errors are statistically independent. The 
channel s/n, y, namely E/N,, is a random variable in a Rayleigh 
fading channel, and its Probability Distribution Function (PDF) is 
given by* 


oe =e, (11) 


where 
I = Efy}. (12) 


The performance of the soft decision demodulation scheme will be 
compared with that of the basic log-PCM transmission system. Ac- 
cordingly, we note that the digital noise power for the log-PCM system 
Grad b 
is 


N 
a= ) PP’ — P)*S,, (13) 

w=) 
where S,, is the sum of the A-factors associated with the number of 
error patterns containing w bit errors in the N-bit words. If the binary 


log-PCM signal is transmitted using CPSK modulation, the bit error 
probability as a function of channel s/n y is’ 
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Ply) = Q(v2y). (14) 


The average bit error probability for the Rayleigh fading channel is 
given by 


= f fly) P(y)dy (15) 


and after substituting f(y) and P(y) into eq. (15), P is substituted for 
P in eq. (18) to yield the digital noise power. We do this on the priviso 
that bit errors occur independently, i.e., ideal interleaving is used.! 


3.1 Fixed soft demodulation threshold system 


We now consider the application of soft decision demodulation to 
enhance the performance of log-PCM transmission over a Rayleigh 
fading channel. The soft decision demodulation system to be consid- 
ered contains a fixed threshold T as described in Section II. We specify 
the value of this threshold as a function of the average s/n, I’, such 
that the erasure zone width of +Z, see eq. (6), becomes 


Z = TVE, (16) 
where the average s/n, I, is related to E by 
EK 
T= Ey} = N, (17) 


and N, is the one-sided spectral density of the additive white Gaussian 
noise in the Rayleigh fading channel. The average bit error probability 
for CPSK with Rayleigh fading is from eqs. (7) and (11): 


p= | re Qv2ydy 





-3)1- Lm (1g) 
“9 V1+Tr] 


We now apply the average process to determine the probability Pp 
of occurrence of an undetected bit error. From eqs. (8) and (16) we 
formulate 


Pp(y) = Q(v2y + TV2P), 
whence, with the aid of eq. (11), 


Pp = ie c eVTQ(V2y + TV2P)dy 


r=) TJor 
= Q(TV2F - Vie lq (fe), (19) 
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The average of P, is 


P, =i =!" P.(y)dy (20) 
and from eqs. (8) and (9) 
P.(y) = Q(V2y -— TV2T) - Poly) (21) 
and, hence, 
P, = | 5 Q(VBy — TV2P)dy — Pp. (22) 


This expression can be simplified to 


P, = 1 - 2Q(TV2P) 


: T | : (=) 
147? 1 — 2Q Gael (23) 


By replacing Pp, P, and Pp = MP, in eq. (1) by the average values 
Pp, P, and P, given by eqs. (19), (18), and (23), respectively, we 
-formulate the digital noise power as 


Qs fe em r r 
é= [Qa \/ tke Q (rar V3) 


M 
{5 Ape Mis" 
I=1 


iS Wy 
2 a 


+ {a — Q(TV2P)] - 














N 
) fa 
l=M+1 


ot 
P a 14r 
1+T 


r 2 
{1 —Q (var \ / +) M6bS?. (24) 


Observe that if only hard decision demodulation is used, the digital 
noise power becomes 


1 1 / N 
e = 9 ( = 1 : =} 2 A). (25) 


3.2 Individual bit soft demodulation thresholds 


Rather than employ the same threshold T for each of the M first 
bits in the N-bit PCM word, we now consider the case when the first, 
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second, ---, Mth bits, are assigned thresholds 7,, T:, ---, Ty, 
respectively. When this procedure is adopted, the digital noise power 
of eq. (24) becomes 


; M 1 ( r N ; M 
—— +-|1- \ / A; + 6S . 26 
: 2 ey 1 Re al wees 2 Bi 28) 
where 
2 [an 0) r 
Qa) = jacrivar) y/ +7 e ova 1+T 


-[A; — 6S?] (27) 











and 
B= 1- Q(T,V2P) 
, / TT - \, / 0 
_ l+T ett -Q (nvr +) (28) 
The terms 


a, + 6S*B; %t=1,2,---,M 


are minimized with respect to the soft decision erasure threshold 7; 
for a given I. The M values of T; found this way are substituted into 
eqs. (27) and (28) to give a, and 8, which in turn are used to estimate 
the minimum digital noise power with the aid of eq. (26). The proce- 
dure is repeated for different values of I’. Thus, for a soft demodulation 
system where each bit is assigned a unique threshold T), the digital 
noise power, and hence overall s/n, is determined as a function of 
channel s/n, I. Observe that our optimization process causes the 
individual thresholds T; to be independent of the value of M for a 
given average channel s/n, I. For example, if we compare two soft 
demodulation systems having M = 3 and M = 6, we find that they 
have the same 7,, T2, and T3. However, the system having M = 3 has 
T, = T; =, -, = Tg = 0, whereas the system with M = 6 has only 
T7 _ Ts = 0. 


IV. PERFORMANCE CRITERION 

We use as our objective performance criterion the overall s/n, 
defined as the power S? of the input signal to the noise power of the 
overall error signal, viz: 


; S? 


SS 
@+eét+eé 


(29) 
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where ¢? and ¢2 are defined in connection with eq. (3), and ¢2 is the 
digital noise power. Although ¢? and ¢? are constant for a given log- 
PCM encoder and for an input signal having a given PDF, ¢? is 
dependent on the transmission and reception techniques and on the 
channel. Its value is given by eqs. (24) and (26), and as described in 
Section III. 


V. THEORETICAL PERFORMANCE 


The logarithmic PCM codec considered in our theoretical calcula- 
tions was 8-bit u-law PCM, uw = 255. The binary code employed was 
binary-folded PCM, where the MSB was the polarity bit of the 
quantized sample, and the remaining bits represented the sample 
magnitude. The range of the quantizer was from —1 to +1, and the 
input signal was assumed to have an exponential PDF, 


eae 
Deh) 35 xp S ’ 


a density function that is known to be representative of the long-term 
PDF of speech signals. We considered two input power levels of —17 
dB and —40 dB, which corresponded to the input signal having a 
standard deviation of V2/10 and 0.01, respectively. The —17 dB was 
the input level to give maximum overall s/n in the absence of trans- 
mission errors, whereas the lower input level of —40 dB had a corre- 
spondence with low-level input speech signals. The single-error A- 
factors employed in our calculations were those derived in Ref. 1. The 
digital modulation was CPSK. We emphasize that our results are for 
the Rayleigh fading channel. 

We now present in Section 5.1 a series of tabulated results calculated 
from the equations provided in Sections III and IV. In Section 5.2 we 
provide additional theoretical results, but this time in a graphical 
format. 


(30) 


5.1 Soft demodulation using individual bit thresholds 


Table I shows the values of the individual thresholds T,, To, --- , 
Ts, as a function of channel s/n, for an input power level of —17 dB 
and a relative mean square error 6 of 0.1. This table relates to the 
individual bit soft demodulation threshold scheme described in Section 
3.2. The nature of log-PCM is that the bit that generates the greatest 
digital noise power when it is erroneously received is the most signif- 
icant magnitude bit. This bit is the second bit in the word, and when 
soft demodulation is applied it has a threshold T,. Thus, because the 
second MSB is capable of generating the largest error it is assigned 
the largest threshold. In general, the higher the contribution to the 
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Table |—Individual bit thresholds, To, as a function of channel s/n, I, for input level —17 dB, 6 = 0.1 


T; 


1.398231 
0.637306 
0.327333 
0.178123 
0.099486 


0.055756 
0.031123 
0.017558 
0.009995 
0.005612 
0.003194 
0.001817 
0.001228 


T 


1.594590 
0.727069 
0.373505 
0.203837 
0.113294 


0.063378 
0.035811 
0.020109 
0.011319 
0.007441 
0.004631 


0.001943 
0.000779 


Ts 


1.431157 
0.653243 
0.335553 
0.182689 
0.101778 


0.056953 
0.032048 
0.017823 
0.011901 
0.005312 
0.002849 
0.001618 
0.001206 


T 


0.687725 
0.313625 
0.161287 
0.087959 
0.048928 
0.027489 
0.015410 
0.008295 
0.004986 
0.002778 
0.001673 
0.001069 
0.000662 


Ts 


0.000000 
0.000001 
0.000001 
0.000003 
0.000001 


0.000007 
0.000001 
0.000009 
0.000061 
0.000035 
0.000057 
0.000046 
0.000160 


Te 


0.000000 
0.000001 
0.000001 
0.000003 
0.000006 


0.000007 
0.000020 
0.000019 
0.000073 
0.000010 
0.000039 
0.000494 
0.000109 


T; 


0.000000 
0.000001 
0.000001 
0.000003 
0.000006 
0.000007 
0.000007 
0.000020 
0.000071 
0.000011 
0.000035 
0.000474 
0.000105 


Ts 


0.000000 
0.000001 
0.000001 
0.000003 
0.000001 


0.000007 
0.000006 
0.000019 
0.000008 
0.000015 
0.000034 
0.000459 
0.000062 


Table I!—Individual bit thresholds, To, as a function of channel s/n, I, for input level —17 dB, 6 = 0.01 


Ti 


2.160955 
0.987638 
0.506489 
0.275695 
0.154197 


0.086858 
0.048570 
0.027583 
0.013697 
0.012576 
0.004466 
0.003976 
0.002335 


T? 


2.327636 
0.901698 
0.547609 
0.298701 
0.166714 


0.093416 
0.052675 
0.029888 
0.015763 
0.008794 
0.004910 
0.003906 
0.002483 


Ts 


2.192461 
0.998271 
0.514840 
0.279855 
0.155108 


0.086822 
0.050988 
0.027333 
0.016203 
0.010522 
0.004599 
0.002797 
0.002307 


T, 


1.582595 
0.721279 
0.371179 
0.202130 
0.112054 


0.062541 
0.035585 
0.019425 
0.010958 
0.007447 
0.004615 
0.001813 
0.000783 


Ts 


0.980817 
0.447031 
0.229991 
0.125606 
0.069540 


0.039325 
0.021794 
0.012291 
0.007052 
0.004531 
0.002283 
0.001078 
0.000809 


Te 


0.308022 
0.140304 
0.072350 
0.039297 
0.021877 


0.012327 
0.006826 
0.003812 
0.002211 
0.001325 
0.000647 
0.000372 
0.000355 


T; 


0.000000 
0.000001 
0.000001 
0.000003 
0.000006 


0.000007 
0.000018 
0.000021 
0.000066 
0.000023 
0.000044 
0.000423 
0.000097 


Ts 


0.000000 
0.000001 
0.000001 
0.000003 
0.000006 


0.000007 
0.000008 
0.000020 
0.000012 
0.000005 
0.000036 
0.000242 
0.000098 


digital noise power by an erroneous bit, the wider its erasure zone. We 
see in Table I that the erasure zones for T; to Tg were very narrow, 
justifying our decision to only perform soft demodulation on the M 
(here 4) most significant bits. For high values of channel s/n, I’, the 
probability of making an error in the bit regeneration process was 
small. Consequently the erasure zones were narrow for all the N-bits. 
As the channel s/n was decreased the erasure zones for the four MSB’s 
widened, and for a channel s/n of 0 dB we observed that the boundaries 
of the erasure zone exceeded +VE. 

When a good corrector was employed such that 6 = 0.01, the 
threshold values increased as shown in Table II, all other parameters 
being those associated with Table I. As the corrections were more 
accurate it was safer to activate the corrector more frequently com- 
pared to when the corrector having 6 = 0.1 was used, and this activity 
is reflected by the larger threshold values in Table II compared to 
those in Table I. 

A poor corrector was associated with 6 = 0.4, and the application of 
such a corrector was to yield a high amount of “correction noise” 
because the replacement of rejected samples was not very accurate. 
The intervention of the poor corrector was less likely to occur if the 
thresholds were reduced, and this was what happened, as can be seen 
in Table III, where 6 = 0.4. 

When the input level was reduced to —40 dB, 6 = 0.1, the threshold 
values were increased (except for the polarity bit), as is apparent by 
comparing Tables I and IV. This occurred because, for the same 
magnitude bit in error, more noise tended to be produced when the 
input power was low. We do not present tables for 6 = 0.01 and 0.04 
when the lower input level of —40 dB was used, but the same trends 
observed in Tables I to III for the higher input were evident at this 
lower input power. 


5.2 Graphical results 


We commence our presentation of theoretical graphical results by 
displaying in Fig. 1 the variation of overall s/n, namely s/n, as a 
function of channel s/n for an input signal level of —17 dB, and soft 
decision demodulation applied to the four MSB’s. The curve for the 
basic 8-bit u-law PCM, i.e., no soft demodulation, provided a reference 
from which the soft demodulation performance can be judged. We also 
display an additional reference curve that relates to a fixed-bit weight- 
ing strategy applied to u-law PCM. In this weighting scheme every /th 
bit in the PCM word has its magnitude multiplied by V¢,, | = 1, 2, 
..., N. Reference 5 presents the method for determining vV¢, for 
CPSK modulation and a Rayleigh fading channel in order to minimize 
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Table I1I—Individual bit thresholds, To, as a function of channel s/n, I’, for input level —17 dB, 6 = 0.4 
T1 T. Ts T4 Ts Ts T, Ts 
0.860252 1.085411 0.900163 0.000000 0.000000 0.000000 0.000000 0.000000 
0.392162 0.494988 9.410554 0.000001 0.000001 0.000001 0.000001 0.000001 
0.201742 0.254390 0.211293 0.000001 0.000001 0.000001 0.000001 0.000001 
0.109656 0.138744 0.115081 0.000003 0.000003 0.000003 0.000003 0.000003 
0.061201 0.077085 0.063834 00.00001 0.000000 0.000000 0.000001 0.000000 
0.034235 0.043139 0.035805 0.000007 0.000007 0.000007 0.000007 0.000007 
0.019297 0.024132 0.020214 0.000002 0.000024 0.000014 0.000026 0.000023 
0.011055 0.013769 0.011485 0.000052 0.000020 0.000020 0.000019 0.000021 
0.005944 0.007815 0.006511 0.000061 0.000070 0.000068 0.000014 0.000004 
0.003554 0.004531 0.003261 0.000093 0.000022 0.000017 0.000005 0.000014 
0.002395 0.002428 0.001588 0.000099 0.000037 0.000035 0.000034 0.000041 
0.001068 0.001738 0.001073 0.000106 0.000235 0.000469 0.000477 0.000473 
0.000619 0.000813 0.000732 0.000118 0.000097 0.000145 0.000098 0.000097 
Table I!V—Individual bit thresholds, To, as a function of channel s/n, I, for input level —40 dB, 6 = 0.1 
Ti T. Ts T4 Ts Ts T7 Ts 
1.397515 2.799741 1.564703 0.907276 0.074134 0.000000 0.000000 0.000000 
0.637817 1.285825 0.714349 0.413849 0.033814 0.000001 0.000001 0.000001 
0.327897 0.659582 0.366937 0.212889 0.017447 0.000001 0.000001 0.000001 
0.178242 0.361774 0.199849 0.115725 0.009486 0.000003 0.000003 0.000003 
0.099536 0.198387 0.110899 0.064638 0.005377 0.000001 0.000006 0.000006 
0.055780 0.108670 0.062466 0.036221 0.002834 0.000007 0.000007 0.000007 
0.013398 0.062405 0.035221 0.020151 0.001512 0.000021 0.000020 0.000021 
0.017505 0.038389 0.019635 0.011423 0.000924 0.000020 0.000020 0.000019 
0.010103 0.023034 0.011127 0.006421 0.000536 0.000012 0.000069 0.000013 
0.004531 0.014624 0.005947 0.003738 0.000420 0.000000 0.000005 0.000006 
0.003000 0.005471 0.004557 0.002030 0.000189 0.000048 0.000037 0.000036 
0.001844 0.005023 0.001552 0.001066 0.000149 0.000472 0.000484 0.000242 
0.001283 0.000775 0.001254 0.000766 0.000245 0.000111 0.000062 0.000097 


WITH 
SOFT DECISION 
DEMODULATION 


OVERALL s/n IN DECIBELS 


~*s WITH FIXED WEIGHTING PROFILE 





0 10 20 30 40 50 60 
CHANNEL s/n IN DECIBELS 


Fig. 1—Variation of overall s/n as a function of channel s/n for M = 4, input power 
level = —17 dB, 8-bit p-law PCM. 


the digital noise power. The curves of Fig. 1 therefore show the gain 
in s/n due to using weighting as compared to the basic p-law PCM, 
and the further gains in s/n that soft decision demodulation provide. 

When a poor corrector corresponding to a 6 of 0.4 was used, we 
achieved a marginally better performance using soft demodulation 
than when only a fixed weighting profile was used. Improving the 
corrector, i.e., reducing 6, yielded greater gains in s/n. While 6 = 0.1 
is relatively easy to achieve, values of 6 < 0.01 require complex 
corrector algorithms. We observe that for 6 = 0.1 our soft demodulation 
strategy of individual soft thresholds yielded a gain of 7 dB and 4 dB 
over conventional and fixed weighted u-law PCM, respectively, over a 
wide range of channel s/n. 

Reducing the input power level to —40 dB resulted in a severe 
reduction in s/n for conventional p-law PCM and for fixed-bit 
weighted y»-law PCM. However, by using soft demodulation the s/n 
was approximately the same as for the —17 dB input level. This can 
be seen in Fig. 2, where M was again 4. For 6 = 0.1 the gains in s/n 
over conventional log-PCM and fixed weighted log-PCM were 19 dB 
and 13 dB, respectively. Figure 3 shows the effect of varying the 
number M of bits subjected to soft demodulation. The input level was 
—17 dB, and the value of 6 was 0.1. Extending M from 2 to 3 is 
worthwhile, but there is no advantage in using M in excess of 4. Thus 
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Fig. 2—Variation of overall s/n as a function of channel s/n for M = 4, input power 
level = —40 dB, 8-bit u-law PCM. 
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Fig. 3—Variation of overall s/n as a function of channel s/n for 6 = 0.1, input power 
level = —17 dB, 8-bit p-law PCM. 
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Fig. 4—Variation of overall s/n as a function of channel s/n for 6 = 0.1, input power 
level = —40 dB, 8-bit u-law PCM. 


our gain in s/n of 7 dB over basic y-law PCM was derived from the 
application of individual soft demodulation on the first three MSBs. 
Observe that when soft decision demodulation was applied to only the 
polarity bit, i.e., the case of M = 1, the performance was worse than 
for fixed weighted y-law PCM. It is important to provide soft decision 
demodulation on both the polarity and most significant magnitude 
bits (M > 2). For the lower input power of —40 dB, the curve for 
M = 1 yielded no gain in s/n, compared to conventional p-law PCM. 
The value of M was required to be 4, as shown in Fig. 4, to achieve 
the maximum gain in s/n. 


VI. COMPUTER SIMULATION RESULTS 


In our simulations we employed four concatenated speech sentences, 
two spoken by males, two by females, that were bandlimited between 
200 and 3200 Hz, and sampled at 8 kHz. These samples constituted 
our input speech sequence. The speech sequence was 8-bit y-law PCM, 
p. = 255, encoded, and weighted where appropriate; two-level CPSK 
modulation ensued. The modulated signal was subjected to Rayleigh 
fading and contaminated by additive white Gaussian noise. The Ray- 
leigh fading envelope C(t) was obtained from a hardware simulator of 
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a frequency-selective Rayleigh-fading mobile radio channel." Differ- 
ent envelope functions were available corresponding to different ve- 
hicular speeds when the propagation frequency was 900 MHz. The 
first M bits in every N bits were initially examined. If the /th bit in 
the kth word satisfied 


C(k)B,(k) + I(k) = | ZI, (31) 
the bit was regenerated according to 
: C(k)B,(k) + I(k) = Z; logical 1 generated 
Bi(k) = ; ; (32) 
C(k)B,(k) + I(k) < —-Z; logical 0 generated 
for 1 = 1, 2, --- , M, where J(k) represented the additive interference 
level. If 
C(k)Bi(k) + I(k) < |Z; ERASURE (33) 


and complete word was rejected. If no erasure occurred during the 
examination of the first M bits, the last (N — M) bits were regenerated 
according to 


3 1c (k)B,(k) + I(k) = 0; logical 1 generated 
*" (C(k)B(R) + I(k) <0; logical 0 generated 


fori=N—-—M,.---,N—-1,N. 

The bit regenerating process, therefore, examined the first M bits, 
and if the input signal level C(k)B,;(k) + I(k) resided outside the 
erasure level for! =1,2, --- , M, the bits were regenerated as prescribed 
by inequalities (82) and (34). Those N-bit words so formulated were 
p-law PCM decoded into samples. However, when an erasure of one 
of the first M bits in any word occurred, the entire word was rejected, 
and consequently there was no corresponding decoded sample. Sam- 
ples so discarded by the soft decision demodulation process were 
replaced by means of prediction and/or interpolation using those 
samples that had been p-law PCM decoded. Various complex predic- 
tion and interpolation algorithms were available to us,!2" but we 
opted for a simple scheme. Knowing that interpolation is superior to 
prediction if there are two correctly received samples adjacent to the 
rejected sample, we replaced the discarded sample x, by means of 
nearest neighbor averaging 


x; = 0.5(X,-1 + X41), (35) 


where xX,-; and X,4; were the decoded samples in the (r — 1)-th and 
(r + 1)-th sampling instants, respectively. When the words at both 
the (r — 1)-th and (r + 1)-th sampling instants were designated to be 
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soft, x, was generated by prediction from the previously accepted 
correct sample, viz: 


x 7. pxr-1, (36) 


where p is the normalized first-order autocorrelation coefficient given 
by 


a EXX,X,41} 
— 2 


Figure 5 shows the variation of overall s/n as a function of channel 
s/n when 8-bit u-law PCM encoded speech was conveyed by two-level 
CPSK modulation over a Rayleigh fading mobile radio channel. The 
input speech level was —17 dB. The threshold values used in our 
experiments were the optimal ones, assuming a particular value of 6, 
and a given channel s/n. Thus Fig. 5 displays a collection of curves 
that use threshold values associated with 6 = 0.01, 0.02, 0.04, 0.1, 0.2, 
and 0.4. The curves shown in Figs. 1 and 5 are in close agreement over 
the usable range of channel s/n, typically when the channel s/n exceeds 
20 dB. Our corrector tends to operate almost exclusively using nearest 
neighbor averaging when the number of rejected samples is small. As 
the channel s/n decreases, eq. (36) is utilized more frequently. Thus 
the 6 associated with our corrector increases as the channel s/n falls, 
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Fig. 5—Simulation of mobile radio channels with a speech input signal. Variation of 
overall s/n as a function of channel s/n for M = 4, input level of —17 dB. 
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Fig. 6—Simulation of mobile radio channels with a speech input signal. Variation of 
overall s/n as a function of channel s/n for M = 4, input power level = —40 dB. 


i.e., the quality of the corrector process decreases with decreasing 
channel s/n. Figure 6 shows the situation when the input speech level 
is reduced to —40 dB, and again we observe a correspondence between 
the curves in this figure and those in Fig. 2. 

Because our corrector has a value of 6 that depends upon the channel 
s/n, employing either interpolation [eq. (35)] or prediction [eq. (36)], 
it is difficult to calculate the optimum thresholds T;. However, over 
the range of channel s/n that yield an acceptable overall s/n, s/n, it 
appears appropriate to use the values of T; that correspond to 6 of 0.1. 


Vil. DISCUSSION 


New theory has been presented for the use of soft decision demod- 
ulation applied to u-law PCM signals transmitted over Rayleigh fading 
channels using CPSK modulation. Each bit in the p-law PCM word 
has been assigned its individual erasure threshold, and these thresh- 
olds have been calculated for high and low input power levels, as a 
function of the channel s/n, and for correctors of differing quality. 
These results are displayed in tabular form in Tables I through IV, 
and in a graphical format in Figs. 1 through 4. In our theory we do 
not specify the corrector algorithms, i.e., those methods of replacing 
samples discarded by the soft decision demodulator, because the num- 
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ber of possible correctors is legion. Instead we calculate the individual 
erasure thresholds for specified values of mean square error power 6 
that is associated with the correction process. We acknowledge that 
this technique does have disadvantages when the actual correction 
algorithm employed is dynamically dependent on which samples are 
being corrected. In this later case it is desirable to adjust the values of 
the thresholds as the corrector algorithm varies if near optimum 
performance is to be maintained. 

From our theoretical deliberations we observed the following salient 
points: 

1. There is no advantage to be gained by applying soft decision 
demodulation to more than 3 or 4 of the most significant bits of the 
8-bit u-law PCM, p = 255, words (see Figs. 3 and 4). 

2. We used two bench markers to measure the performance of our 
soft decision demodulation scheme. These were the basic p-law PCM, 
and the fixed-profile weighted p-law PCM. At the high input power 
level of —17 dB (which corresponded to a peak s/n in the absence of 
transmission errors) the gain in s/n due to soft decision demodulation 
using 6 = 0.1 over basic u-law PCM and weighted y-law PCM was 7 
and 3 dB, respectively, when the channel s/n was 30 dB. Observe that 
with the soft decision demodulation the s/n was 30 dB, a value 
commensurate with toll-quality speech. For the lower input level of 
—40 dB, a level that has some correspondence with unvoiced speech, 
the corresponding gains in s/n were 18 and 12 dB, respectively. Figure 
4 shows that soft decision demodulation is particularly important at 
the low input level, achieving an s/n of 28 dB for a channel s/n of 30 
dB. Indeed, the s/n deteriorated by only 8 dB, the same value observed 
for the higher input level, when the channel s/n decreased from the 
ideal channel condition (~60 dB) to the lower channel s/n of 30 dB. 

3. Provided M = 4, soft decision demodulation always gave a gain 
in s/n compared to our reference systems, even when 5 = 0.4. The 
ability that soft decision demodulation has to maintain its performance 
for both high and low input levels is strikingly demonstrated in Figs. 
1 and 2. Observe that improving the corrector’s performance by a 
factor of ten compared to 6 = 0.1 caused the s/n to increase by 5 dB 
when the channel s/n was 30 dB. 

4, Four concatenated speech sentences were used in our simulations. 
The speech was p-law PCM encoded prior to CPSK modulation, and 
the Rayleigh fading channel was obtained from a real-time simulator. 
The results, cisplayed in Figs. 5 and 6, demonstrate the advantage of 
using soft decision demodulation. We observed gains in s/n of 6 and 4 
dB compared to the basic y-law PCM and the fixed weighted p-law 
PCM, respectively, when the input power was —17 dB and the channel 
s/n was 30 dB. The corresponding gains when the input power was 
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~—40 dB were 17 and 9 dB, respectively. Although the corrector used 
in our simulations could switch its algorithm [see eqs. (35) and (36)], 
for the gains in s/n quoted above eq. (35) was used most of the time, 
and 6 was approximately 0.1. 

From our theoretical results and simulations we conclude that the 
application of soft decision demodulation, where each bit in the p-law 
PCM word is assigned its unique threshold value, does offer significant 
advantages in s/n when the transmission is by CPSK modulation over 
Rayleigh fading channels. The gains in s/n are achieved by only a 
marginal increase in complexity at the receiver, while the transmitter 
is unchanged. Careful optimization of a combination of soft decision 
demodulation at the receiver with a weighting strategy at the trans- 
mitter leads us to expect further improvements in s/n, although at the 
expense of added system complexity. 
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Linear Equalization Theory in Digital Data 
Transmission Over Dually Polarized Fading 
Radio Channels 


By N. AMITAY and J. SALZ* 
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A theory of linear least-mean-squares equalization in digital data commu- 
nications operating over two coupled linear dispersive channels, with partic- 
ular application to dually polarized terrestrial radio systems is presented. We 
jointly optimize transmitter and receiver matrix filters when the inputs are 
two independent quadrature-amplitude-modulated data signals of fixed total 
average power. Formulas for minimum total mean-square error, upper bounds 
on probability of error, matched filter bounds, and the Shannon limit are 
provided. Using this theory in conjunction with a proposed propagation model 
for the dually polarized radio channel, we provide estimates of probability 
distributions of the data rates that can be supported by the optimum receiver 
structures as well as the jointly optimized transmitter-receiver. The distribu- 
tion of data rates predicted from the Shannon limit as well as from the 
matched filter bounds are also given. For the dually polarized radio channel, 
we find that linear equalization structures can practically eliminate the effects 
of cross-polarization interference contributed by the antennas and their feeds. 
At very low outage probabilities, the combined-channels data rate achieved 
with the jointly optimized structures is about 4 b/s/Hz less than predicted 
from their matched filter bounds. Also, the combined data rate associated with 
the Shannon limit is about 3 b/s/Hz higher than predicted by the matched 
filter bound at very low outage probabilities. Sensitivity analyses reveal that 
about 1 b/s/Hz can be gained for each 3-dB increase in carrier-to-noise ratio. 
For each order-of-magnitude decrease in the required error rate, about 0.3 
b/s/Hz is given up. 
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I. INTRODUCTION 


Communications via dually polarized radio channels is an effective 
means of reusing existing bandwidth. It is well known that one can 
transmit and receive two orthogonally polarized waves, e.g., linear, 
circular, or elliptical. Information can be conveyed by independently 
modulating the signals carried by each of these waves. In conventional 
terrestrial microwave radio systems, linear orthogonal polarizations— 
i.e., horizontal and vertical—are usually employed. 

Digital communications over a single radio channel (single polari- 
zation) is adversely affected by multipath fading, which causes Inter- 
symbol Interference (ISI). In dually polarized digital radio communi- 
cations, one also has to contend with Cross-Polarization Interference 
(CPI). This interference, which is dispersive in general, is due to 
antenna misalignments, imperfect waveguide feeds, fading, and other 
propagation path anomalies. In order to utilize the dually polarized 
radio channels to their maximum capacity, it is necessary to mitigate 
the deleterious effects of ISI and CPI. 

Various approaches to this problem have been suggested.'© Some 
of them address the problem of CPI cancellation only, some minimize 
CPI and noise, and others treat nondispersive channels only. Often 
these methods are based on cross-channel frequency response esti- 
mations, thereby facilitating at least a partial cancellation of CPI. The 
chief purpose of this paper is to establish a sound theoretical base for 
realizing effective algorithms and devices for the simultaneous and 
optimal compensation of cross-polarization and intersymbol interfer- 
ences in the presence of noise. 

The dually polarized channel is modeled as a 2 x 2 complex matrix 
with frequency-dependent elements followed by an additive noise 
vector. The diagonal entries represent the cochannel (in-line) fre- 
quency transfer characteristics, while the off-diagonal terms represent 
the cross-channel transfer characteristics. The four entries in this 
matrix channel can be viewed as a sample from a random collection 
of frequency functions. The presence of noise implies that each matrix 
channel in the collection is limited to a maximum data rate, for a 
given error rate and specific communication method. Because of the 
random nature of the matrix channel, a meaningful performance 
measure for a particular communication method is the probability 
distribution of data rates that can be supported at a certain error rate 
objective. 

Figure 1 depicts the problem considered. Two data sources are first 
passed through a matrix transmitter filter. The diagonal entries con- 
trol the cochannel signal shaping, while the off-diagonal terms control 
the cross-channel signal shaping as well as a possible power distribu- 
tion (transfer) between the two channels. The two emerging output 
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Fig. 1—Dually polarized radio transmission. 


signals, represented by a vector, are then transmitted over the matrix 
channel. At the receiver, independent noise sources are added, and the 
resultant vector signal is processed by a linear matrix fflter, which 
acts as a linear equalizer/canceler. 

We pose these general questions. How does one choose the receiver 
matrix filter so as to minimize a reasonable cost function? Next, how 
does one choose an optimum transmitter matrix filter under a reason- 
able power constraint? Finally, how does the performance of these 
optimum structures compare with the ultimate performance predicted 
by information theory (Shannon limit)? 

After a discussion of the channel model in the next section, we 
describe the modulation and the transmitter models in Section III. In 
Section IV we discuss the receiver optimization problem, while in 
Section V we derive the theory for joint transmitter-receiver optimi- 
zation. The Matched Filter (MF) bounds on performance are derived 
in Section VI, and in Section VII the information theory limit is 
developed. Description and discussion of the propagation model ad- 
vanced in this paper can be found in Section VIII. In Section IX we 
give numerical evaluations and discuss the results. Section X contains 
our summary and conclusions. 


Hl. THE CHANNEL MODEL 


A general linear, baseband-equivalent, dually polarized transmission 
channel can be characterized by the matrix impulse response 


ai) = i oa (1) 
Cio(t) C22(t) 


or its Fourier transform 


C( oa ie e(t)e“dt a eS aoe (2) 
eo Cio(w) Co2(w)/ 
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which is the frequency transfer characteristic of the transmission 
medium. The diagonal entries in the matrix c(t) are the complex- 
valued impulse responses of the co-polarized channels, while the off- 
diagonal entries are the complex-valued cross-polarized channel im- 
pulse responses. To complete the channel characterization, we assume 
that the noises at the two output ports are additive white Gaussian 
processes represented by the complex vector, 


(t) 
v(t) = (" (3) 
v(t) 
and characterized by the covariance matrix 
E[v(ts)v' (t2)] = Nod(t: — te)l, (4) 


where »;(t) and v2(t) are baseband-equivalent, zero-mean, complex 
Gaussian processes with common double-sided spectral density, No. 
The notation, E[-], stands for mathematical expectation, 6(-) is the 
Dirac delta function, ' stands for conjugate transpose, and I is the 
identity matrix. This channel model is illustrated schematically in Fig. 
Zi 


Il. MODULATION AND TRANSMITTER MODELS 


Our investigation focuses on the class of efficient linear modulation 
methods known as Quadrature Amplitude Modulation (QAM). A 
convenient representation of these modulated signals is 


s(t) = Re{(exp(i2nf,t) LY ang(t — nT}, (5) 
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Fig. 2—Linear channel model for dually polarized radio transmission. 
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where Re(-) stands for the “real part” and f, is the carrier frequency. 
The data symbols, {a,}, transmitted at T-second intervals take on 
values on a two-dimensional lattice and are therefore complex valued. 
In general, the pulse g(t) is also complex valued. 

In a dually polarized communications system, two independent 
QAM data signals as individually represented in (5) are transmitted, 
one on the horizontal channel and the other on the vertical channel. 
In this situation, the transmitted data signal is modeled as a two- 
dimensional vector 


Vi (t) 
V(t) = ( = Re lexw(xf0 > p(t - nt)Ad} ; (6) 


V2(t) 
where 
an Qnr + 1Ant 
wes) 
bn Dnr = ig tbat 
is the nth pair of data symbols and p(t) is the transmitter matrix filter 
impulse response, 
Pit) par(t) 
p(t) = ( : (8) 
Pi2(t) Po2(t) 


The diagonal entries in (8) are cochannel transmitter impulse re- 
sponses acting as signal shapers, while the off-diagonal impulse re- 
sponses act as cross-channel signal shapers as well as power distribu- 
tors. 

For two independent input data streams, the covariance matrix of 
the data symbols is diagonal, 


E|a,|? 0 oa 0 
E{A,A}] = = ; (9a) 
0 E|b,|? 0 «@& 


and E[A,A!] = 0, n ¥ m. For a rectangular lattice consisting of odd 
positive and negative integers, 


a 2(# = ‘), (9b) 





3 


where L? is the total number of points in the constellation. We now 
refer to the cochannels as channel “a” and channel “b”, respectively. 
By use of (8) and (9), the average transmitted power in each of the 
channels is, by a straightforward calculation, 


2 foo] 
P= a [ (| pir(t) |? + | por (t) |?)dt 
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and 


2 
Py = z i (| Poo(t) |? + | pie(t) |?)dt. (10) 


The total average transmitted power is, therefore, 


2 foe] 
Py = P, + Py = 7 { trace[p' (t) p(t) |dt 


trace[P' (w)P(w) |dw, (11) 


oa lL 
T 27 J- 
where P(w) is the Fourier transform of p(t). The quantity P,, in (11), 
the total average power, is constrained to be fixed throughout this 
work. 
IV. THE OPTIMUM RECEIVER PROBLEM 


Let H(w) be the overall matrix transfer function of the transmitter 
filter in cascade with the channel C, 


H(w) = C(w)P(w). (12a) 


Consequently, the equivalent baseband signal plus noise vector at the 
receiver can be written 


S(t) =} H(t — nT)A, + v(t). (12b) 


If W(t) denotes the impulse response of the receiver matrix filter, as 
shown in Fig. 1, the output vector signal is then represented as 


S,(t) = 3} i W(t — r)H(7 — nT)A,dr 


{ (") 
+ W(t — 7)v(r)dr = (13) 
ai S20 (t) 


and the sampled vector output at t = 0 is S,(0). This is a representative 
sample since it possesses the same statistics as the nth sample because 
of stationarity. 

Next, we define the error vector e« as the difference between S,(0) 
and the vector data symbol A,, 


0) —a, 
meen 0° ) 
S20(0) — b 


and we form the average-squared error matrix, 


£ = Efee'] (14) 
MSE = MSE, + MSE, = trace &} ° 
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The diagonal elements of £ are the individual mean-square errors of 
the two channels, MSE, = E|s19(0) — a,|” and MSE», = E|s99(0) — 
b, | ?. Substituting (13) into (14) and making use of (4) and (9), we get 


f= o4 {1 _ i W(-—7)H(r)dr — ‘a [W(—7)H(r)]'dr 
+y | W(-—7)H(r7 — nT)dr { [W(-7r)H(r — nT)}'dr 


| Win) Wondr (15) 


It should be noted that MSE, is a function of W1; and Wo, only ~ 
while MSE, is a function of Wiz and Wo: only. Our objective is to 
minimize MSE, expressed in (14), with respect to the elements of the 
filter matrix W, i.e., to determine the elements Wj); j, / = 1, 2, which 
minimize MSE in (14). From the previous discussion it is clear that 
MSE; and MSE; are independent of one another. Therefore, minimi- 
zation of (14) is tantamount to the independent minimization of MSE; 
and MSE». 

The justification for using the quadratic cost function, MSE, is due 
to its mathematical simplicity and tractability. Moreover, as will be 
seen later, minimizing this cost implies minimizing a reasonable upper 
bound on error rate. Additionally, its use is practically motivated, 
since it lends itself to an easy estimation method often used to update 
adaptive equalizer tap coefficients in practical systems. 

Since trace ¢ is a positive quadratic form in W(t), we set its gradient 
equal to zero and are thus guaranteed to obtain a global stationary 
point. Proceeding with the calculus of variation, we replace the matrix 
W by 


e10Wi €216 Wo) 
ot (16a) 
€1260 Wie €226 Woo 
and set 
0 0 
1 mt 
0€11 eee de ee = § (16b) 
0 0 


re) 6] 
—— trace —— trace 
O€12 5 O€22 é 
at €1) = €21 = €12 = €2 = 0. 
After considerable detailed calculations, summarized in Appendix 
A, we get an integral equation for the optimum matrix filter, W,(t): 
il 1 
W.(-7) = = A" (7) - 5 Y UH" (7 - nT) (17a) 
o eG 
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with 


U, = { W.(-t)H(t — nT )dt, o” = N,/o%. (17b) 


Equation (17) can be interpreted as a representation of the optimum 
filter, W,(t), by a matrix matched filter, with impulse response H‘(—t), 
followed by a sampler at t, = nT and a matrix transversal filter, or a 
nonrecursive matrix digital filter, with matrix tap coefficients U,,. The 
matrix representation in (17) is symbolically similar to the scalar 
case,!” with matrices playing the role of scalars. When the channel 
matrix impulse response (1) is diagonal, i.e., cross-polarization inter- 
ference is absent, (17) reduces to two scalar equations representing 
the two optimum cochannel equalizers. In the general case, (17) 
represents a cross-coupled transversal structure admitting equaliza- 
tion as well as cancellation. 

When (17) is postmultiplied by W'(—t), integrated, and then sub- 
stituted into (15), we get 


&= oF E _ ip w-DH Oat] = o3(I — U,) (18a) 


and 
MSE, = Min trace{é} = trace{t[W,(t)]} = trace{é}.  (18b) 


Furthermore, explicit equations for the optimum matrix tap coeffi- 
cients, U,,, are obtained by postmultiplying (17) by H(r — kT), for all 
k, and then integrating. The result is a set of linear equations in the 
matrices U,, 


o U, = R; = >; UnRr-n, all k, (19) 
where 
R, = ‘) H'(r)H(r — kT)dr = Rt,. 


The discrete matrix convolution equation (19) has a simple unique 
solution obtained by taking the Fourier series of both sides. Defining 


U(w) Gi onT 
= ea 20 
(oe 2 (.) ° - 


Uf. IT (Uw 
( )= Le | ( ( elon dy (21) 
R, 2a —x/T R(w) 
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with 


and substituting (21) into (19), we obtain 
U(w) = R(w)[o? + R(w)]} 7. (22) 


Again, (19) and (22) are strikingly similar in structure to the case 
obtained in the scalar situation,” with the proviso that matrices play 
the role of scalars and with the obvious condition that the matrices 
do not necessarily commute. 

We now express (18) more explicitly by obtaining U, from (21) and 


(22) 
a/[T -1 
pags [r+ A] dw 
oO 


Qa J-x/T 





and 
MSE, = trace{é,}. (23) 


The connection between R(w), given in (20) and (21), and the matrix 
channel transfer function H(w) is made by equating R, from (21) to 
the definition in (19), 


00 T a/T 
k= { H'(7)H(r — nT)dr = — i R(w)e™“"" dw 
—0 Qa ~x/T 


1 [°° 
=— { H"(o)H(w)e-*" de 
Qa —00 
i Ce Qnk Qnk\ _, 
=_- — Tt as eee _ oa |) -iwnT 
yan eee x H (. T H (. T e dw. (24) 
Equation (24) implies that 


Rw) = EH (0-44) w (0-4), * (25) 


This is recognized as the aliased, or folded, matrix spectrum about the 
Nyquist frequency 7/7. For the nonexcess bandwidth transmission 
case, 1.e., 

H(jw) =0 for |w| 2 2/T, 


eq. (25) reduces to 
R(w) = 7 H1 (w)H(w). (26) 


At this point we introduce simplifying normalizations and perform 
some sanity checks. Focusing attention on the nonexcess bandwidth 
case with constant and identical transmitter filters, i.e., 


P(w) = Plz) = 6 ae 


lw| <2/T or lzl <5 with H = KC(w), (27) 
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where the normalized frequency variable is 








wae a w =fT, (28) 
2a 
we find from (11) that the average power per channel is 
Py = Py = Ko3/T? = 2,72: (29) 
The quantity 
No N./T K? 
T= ie / Ke=—, (30) 
oF P, p 


where p is the clear air carrier-to-noise ratio (CNR) in each channel, 
being typically 2 x 10°, or 63 dB. Applying the normalization defined 
in (26) through (30), we rewrite (23) as 


-1 


1/2 
f= 65 { [J + pC*(z)C(z)]"'dz 
/2 
and 
1/2 
MSE, = «3 J trace[I + pC*(z)C(z)]"'dz. (31) 
-1/2 


Again, it is evident that when H in (27) is diagonal (no cross- 
polarization interference), (31) reduces to the scalar case.'® As we 
expected, when co” —> 0, (p — ©»), MSE, — 0; and when a”? > , 
(p — 0), MSE, — 203. 

In the next section we use the closed-form expression for MSE, in 
(31) to optimize the transmitter matrix filter. 


V. THE TRANSMITTER OPTIMIZATION PROBLEM 


For the general transmitter matrix filter, P(w), the optimization 
problem has proved thus far to be intractable. This is in contrast to 
the scalar case where a general solution exists.’®> However, for the 
nonexcess bandwidth case [i.e., where there is only one term in the 
sum (25)], we have been able to arrive at a complete solution. 

We begin our analysis by substituting (26) and (28) into (23) to 
obtain 


1/2 
f= 04 4) : [I + P*(z)C*(z)C(z)P(z)/o?T] dz. (32) 


Our problem now is to minimize MSE, = trace[£,] over all matrices, 
P, in the class of ideally bandlimited filters. More explicitly, we wish 
to determine P,, which minimizes MSE, subject to an average power 
constraint, 
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1/2 


MSE,, = Min { trace(&,[W., P])dz 
P -1/2 
1/2 
= { ; trace(é[W., Po])dz (33) 


1/2 
with ‘ trace[P'(z)P(z)]dz = constant 
-1/2 


The search is for P,, the class of filters that optimally distribute the 
fixed available average power between the two channels as well as 
across the frequency band of each channel. 

We observe that (33) in conjunction with (29) and (80) leads to the 
matrix minimization of the functional 


Min[trace{(I + p,P'CtCP)7! + XP'P}], (34) 
P 


where \ is a Lagrange multiplier to be determined from the power 
constraint and p, = p/2K?. 

For each value of z, the matrix C'C is Hermitian and can be 
diagonalized by a unitary transformation y, 


Ci'c=ytay, (35) 


whose nonnegative eigenvalues are 


(Od 
A= ‘ 
0 ro 





Defining 
G=yP 
Pty'tAyP = GTAG (36) 
and observing that for 2 x 2 nonsingular matrices 
_,  trace[-] 
1 — = — 
trace[ - | Seen trace(AB) = trace(BA), 


we express (34) in an equivalent form, 
Min[trace{(I + p.G'AG)! + Ap,GG"}] 
G 


= Mi 2 + trace(QA) 
= 5 1 + trace(QA) + det(QA) 


where Q@ = po.GG' and A = X/po. 

Careful inspection of (37) reveals that since Q is Hermitian and 
positive definitive, and a nonzero off-diagonal entry in Q affects only 
the determinant and not the traces, the minimizing matrix Q must be 
diagonal. 


+X trace(Q)], (37) 
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Therefore, denoting 


qu 0 
Q = 
0 dae 
with qi, = 0 and qx. 2 0, the original minimization problem now 
reduces to a much simpler one, namely, 


1 1 
Mn ag a oe ea 38 
911,922 ine +1 DegGoo+1 (qu ea) (38) 


91192220 


which, with the power constraint 2p = trace[J!?. Qdz], immediately 


yields 
gi. = Max fo we — 1} />s 
(39) 
don = Max fo \/% = 1] /r. 


The procedure for obtaining \, qi, and goz can be found in Appendix 
B. 
We now write the closed-form expression for MSE,, of (33) as 


oe 1 1 
M 0 = 2 i: ————————— + Sina ee at an d. . 
SEup = 04 -1/2 ; +qurAr 1+ 7 ) 


Recall that Q = p,GG' and G = WP. For the optimum Q, we can 
therefore write 


Q=pwP.Piyt and P,P} = V'(QY?/Vp.)(Q*?/V pov 
gin ee 0 - (41) 
NO Van 


Equation (41) yields the functional form of the optimizing transmitter 
matrix filter, P,, given by 


1 
P, = 7 ¥'Q’?S;  S being any unitary matrix. (42) 
Po 


Since S in arbitrary, there is an infinite number of matrices, P,, that 
would give the same MSE,,. The particular choice of S would deter- 
mine MSE, and MSEp, the diagonal entries in &(W., P.), albeit their 
sum, MSE,p, is the same. 

When S = J, P'C'CP = Qa is diagonal and 
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_ (MSE, 0 
£(W,, P,) > ( 0 i) ’ 


ie 1/2 - (48) 
dz i dz ( 
_ 2 ee Sn = ¢g2 > 
MSE = of be 1+qur ’ MSE» = o@ -1/72 1 + qdoode 


From (18a) we obtain U, as 


Zen aelt _ {1 - MSE,/o3 0 
U, I 03 £,(W., PF) = ( 0 1 = MSE, /o? . (44) 


As we shall later see, U, is an important quantity in the determination 
of the system error rate. 
When S = y, P, is Hermitian. In general, however, 


iSeeee i M1 + [Su/?gudr + [Sio|?aaado 4, 
-1/2 (1 + giurAr)(1 + qo2d2) (45) 


2 as es | Sor |? quid + | S22 | 7q22A2 
MSE, = of SS ee 
-1/2 (1 + qirAi)(1 + qe2dz) 


are real quantities, and so are the diagonal entries in the matrix U,. 
The matrices £(W., P,) and U, will not be diagonal in general. 
An interesting case arises when 


s=+(} i); [Sul?=5; 4 1=1,2 46) 
since one can associate this matrix with the transmission of circular 
polarization. Consequently, from (45) and (46), MSE, = MSE.. Thus, 
if we desire to have identical MSEs when the two channels are 
different, a power conditioning filter, y'Q”*/Vp, in (42), followed by 
a circular polarizer in the transmitter will achieve it. 

It is instructive at this point to investigate certain limiting cases. 
For the no cross-polarization case, 


dz 


A O 
y=TI and cto = ( ). (47) 
O re 


From (381), (38), (39), and (40) we can write 
il sc (a | 1 
_ 2 
MES Sg -1/2 ; + pry x 1+ --| a 


1/2 1 1 
MSE,» = a ——_—_— + — —|d 
f “3 —1/2 ; + quai 1+ —| ‘ 


It is therefore possible to view qi; and qz2 as equivalent CNRs in the 
latter case. As long as the Lagrange multiplier \ obeys 








(48) 
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A< Min[A;(z), A2(z)], 


z[<- 
| Is5 


we obtain (see Appendix B) from the power constraint 


1/2 
{ [1/Va1 + 1/Vraldz 
Vi = ae eee (49) 
2p ze { [1/A1 =F 1/reldz 
-1/2 


and 


1/2 2 
| ‘i - [1/Var + 1/Falee 





MSE,, = 63 ———>y-—-— (50) 
2p + 1 (1/A1 + 1/d2)dz 
-1/2 
For nondispersive fades, (48) and (50) reduce to 
1 1 
MSE, = a3 + a 
- 1+pr\; 1+ pdrz2 (51) 


2 [/VAr + 1/Va)? 


MSE,, = 
SEop Of 95 41 A Ne 


As can be verified, when p — ©, both MSEs — 0. For identical fades, 
i.e., \1 = Ao, we get the expected result 


20% 


1+ pr 





MSE, = MSE,, = 


For independent fades with Vy «& Ve and sufficiently large CNR, 
i.e., op — ©, the improvement in mean-square errors is 


fn MSEo _ 2/1 
poe MSEy  1/A1 





This is due to power transfer from the good channel to the bad one. 

For two identical fading channels, the transmitter optimization is 
tantamount to power redistribution within the individual channels 
without power transfer between the channels. The asymptotic im- 
provement in MSE, utilizing an optimized transmitter filter, can be 
calculated as 
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MSE, 
we MSE,, 








1/2 o dz 1/2 ae 
aap +2 -1/2 i -12 1+ pry ie y 
= him tag a Eee a 
a iia ay ie ay 
J = —1/2 Gy 


by Schwarz’s inequality. 


Vi. THE MATCHED FILTER BOUND 


A useful bound on the performance of equalizers can be obtained by 
assuming that one can detect each data vector symbol optimally by a 
matched matrix filter without incurring the penalty of intersymbol 
interference. Clearly, it is impossible for any equalizer to perform 
better than this fictional system. 

We begin by noting from eq. (17) that in the absence of intersymbol 
interference the optimum equalizer filter W,(—t) satisfies the equation 


Wo(-t) = H(t) — 5 UH 


ae 


52 L - Us )H*(t). (52) 


This is recognized as a matched filter followed by a multiplicative 
matrix coefficient. 
The optimum mean-square error matrix is, again, from (18) 


£, = 04 (I — U,), (53) 
where in this case, U, satisfies (19) for k = 0, 
= (I vee U,)Ro 


and 


ay 
£, = 04 [r+ * 
o 


(54) 
MSE, = trace[£,] 
For bandlimited transmission 
ro) x/T 
k= ‘ H'(r)H(r)dr = a i} H' (w)H(w)dw 
—co Qa —1/T 
1 1/2 
== i Pt (z)C' (z)C(z)P(z)dz. (55) 
T J-1/2 
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Consistent with our previous notation in (27) through (30) and the 
definition of R, in (55), the average-square error matrix &, is given by 


1/2 -1 
t, = 04 [r+ p { ) c'Cae| (56) 
—1/2 


1/2 
{ CtCdz = os el 
“1/2 My M2 


where Mi. = Msy, we obtain 


Writing 


‘ Mi Ma\|"* 
Eo = 0d T+ p . (57) 
Mi Moe 
The individual and the total mean-square errors are thus given by 
1+ pM. 
MSE; = 63 = OOOO 
; "4G + pMi:)(1 + pM22) — | pMa|? 

MSE, = ae » 2 


“q + pMi1)(1 + pM22) — | pMai|? 
MSE, = MSE, + MSE, 
As before, it is again possible to optimize the transmitter matrix 
filter P. The problem at hand, while it appears similar to the one 
already solved for the general case of cross-polarization and intersym- 


bol interferences, is different and cannot be deduced from the general 
solution. Here the problem reduces to determining P such that 


1 1/2 = 
oztrace \{r+ sel Pract actePtei| } 


is a minimum, subject to . (59) 
1/2 
trace | { Pre Prene| = constant 
-1/2 


Note that the matrix A = J‘. P'C'CPdz is semidefinite Hermitian, 
and as before, it can be represented as [eq. (36)] 


1/2 
A= A G'AG dz, 
-1/2 
where G = yP. 


As we have already argued in Section V, minimizing (56) is equiva- 
lent to maximizing 


1/2 
Max det { GtAG | (60) 
P -1/2 
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subject to trace J G'G = constant. Since the integrand of A is Hermi- 
tian and positive, the maximization is equivalent to requiring that the 
off-diagonal terms of f[ G'AG be zero. This immediately leads to the 
conclusion that both G and Q are diagonal, Q = G'G. 

Thus, the equivalent minimization problem now reduces to 


; 1/2 ae 
Min trace [1 + = { ; Quar| 
2 


2 
911922 oT J 


1 
ie ay 
1+—— 

o°T J. qudidz 


= Min trace (61) 


911922 


1 


1 1/2 
1+ TT J. Goo Agdz 


subject to [ufo (qur + G22)dz = (Pa + Ps)/(03/T?) = Pay/(o3/T’). 
The functions qi,(z) and qg22(z), which minimize (61), would maxi- 

mize the following integrals: J. qii(z)A1 (z)dz and S12 qo2(z)d2(z)dz, 

subject to the same power constraint. This is a straightforward opti- 


mization problem resulting in 
qu(z) = x6(zi1) and q22(z) = yo(ze); x, y BO, (62) 


where z, and 2, are the normalized frequencies where the eigenvalues 
\, and dg attain their respective maximal values, \1,, and \em. Physi- 
cally, these results imply that power in each band should be concen- 
trated at the frequency of peak transmission. Again, 6(-) is the Dirac 
delta function and the positive constants x and y have yet to be 
determined. 

Substituting (62) into (61) yields 


1 1 
Min MSE, = Min «3 | + ——_|_ 63) 
x,y20 x,y20 
Se eee 


subject to x + y = constant. 
Carrying out the above minimization, we find that 


pe = a= J Peo iam + (Rim — VRan)/-VKim on] 
Im 2m 


and 


p= = pv = Gn Aen reel 468) 
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The quantities p; and p2 can be interpreted as the equivalent CNRs in 
the respective channels. Their sum is constant and equal to 29. The 
conditions for x and y to be nonnegative can readily be verified to be 








—— 1 1 
2 nN m = ae f—— 9 r m = r m 
i : V Aim dom : ; 
and 
2pv im z= eons —~ aids, Aim z= Agm- (65) 
dom im 


Using (61) and the optimized values of x, y given in (64), we finally 
get for the minimum mean-squared errors 


2 Woe ea 


MSE, = ——*.— = 
; i = PiAim -« 2pAimAgm of Rin 7 Nom 
; V nN m m m 
MSE, - _ God = 2 1 (VAq + Vo ) (66) 


1 “f p2A2m ne 2pXimA2m a Nie + om 
( Vv Aim + Vv Nani)? 
MSE,, = MSE, + MSE, = «8 ———__—_- 
: : as 2pA1mA2m Fe Aim — Nom 


As can be seen, for nondispersive fades where there is no intersymbol 
interference (66) is identical to (51). 


VII. INFORMATION THEORY LIMIT 


Here we discuss the information theoretic capacity of dually polar- 
ized radio channels. This quantity is the maximum number of essen- 
tially error-free bits per cycle that can be attained for a given matrix 
channel transfer characteristic and a given clear air CNR. The problem 
is to calculate the Shannon capacity of a multi-input, multi-output 
linear dispersive channel with additive Gaussian noise. This general 
problem has been considered by Brandenburg and Wyner.’® Recently, 
Foschini and Vannucci” called our attention to some technical results 
that make the evaluation of this quantity in our case relatively simple. 

We first recall the capacity formula for a scalar dispersive channel 
with frequency transfer characteristic C(f ) 


c= J) tos {1 +2 1Pniiccyi'f a, bis (67) 
Ww o 


where oa? is the average signal power at the input to the transmitter 
filter P(f), and N, is the double-sided spectral density of the additive 
Gaussian noise. The transmitter filter P(f) is selected to maximize C 
subject to an average transmitted power constraint. 
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In dually polarized communications, the channel frequency charac- 
teristic is a 2 x 2 complex matrix. To facilitate the calculation of 
capacity, we use the polar factorization theorem”: to represent C(f ) 
as 


C(f) = U(f)VCNPAIC(F), (68) 


where U(f) is unitary. (This is the key idea suggested by Foschini 
and Vannucci.”’) Moreover, (35) is used to obtain the representation 


VCNAIC(f) = Vay, (69) 
where 
V1 0 
“= ( 0 ) 


Since the product of two unitary transformations is unitary and 
since unitary transformations do not alter either the total average 
transmitted power or the noise power, the original channel can be 
represented mathematically as two parallel channels, each with fre- 
quency characteristic \1/"(f ) and \37(f ), respectively. This is evident 
from the representation expressed in Ref. 69. 

Since the mathematically equivalent representation is comprised of 
two parallel scalar channels, the capacities add and consequently the 
total channel capacity is 


1 at 2 
Cc =— In {r+ dipun sf a 


In 2 Jw, 


1 { os 3 
+25 Jy, {1 + a | Pol FI tA} df, b/s, (70) 
where P, and Pz are the filter characteristics that optimize C subject 
to a total average power constraint 


Pyy = 203 i |Pi(f) Pdf + 203 aN \Pof)I2df, (71) 
WwW, W2 
where o% and o3 are the average input powers to the transmitter 
filters. 

In our applications, the channels are assumed to be strictly band- 
limited to the band W = (1/T), and therefore, it is convenient to 
normalize the capacity formula by dividing (70) by W. We denote the 
normalized capacity—i.e., the efficiency index, sometimes called the 
Shannon limit—by I, in units of b/s/Hz. 

The total CNR obtained via the constraint (71) is 


2p = P,/2N, W, 
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and, therefore, (70) and (71) can be rewritten in the normalized 
notation as 


1 Ae o% 
sas eT. 2 
I, In 2 | - In + N, | Pi (z) | wo) dz 


1/2 9 
+ i In F oe [Peet ie} _ (72) 
-1/2 No 
a2 1/2 o2 1/2 
with 2p = Ae es | P,(z) |?dz + ae ” | Po(z) | 2dz 


The mathematical optimization can thus be cast as finding 


rr eee 
Max |I,[P:, P2] — A N. |Pi]- + N. |Pol*it, (73) 


|P,I?,|P2|?=0 


where \ is a Lagrange multiplier. Using the calculus of variation on 
(73), we obtain the following two equations 


| Pi(z) |? = Max fo. Mes al 


N,\j1 1 
2-™M Se a ee 


The Lagrange multiplier is evaluated from the power constraint in 
(72), 


ee 1 1 | 
2p = in Mos {0 F _ +| + Max {o F -_ ah dz. (75) 


The iterative procedure outlined in Appendix B can accommodate 
the evaluation of \ in this case, and the proper intervals Z, and Zo. 
These are the intervals over which | Pi(z)|? > 0 and | P2(z)|? > 0, 
respectively. We can now calculate the Shannon limit, I,, from (74) 
and (72) as 


1 At do 
waco l Lm ft] ece fm fl ech (76) 


Vi. PROPAGATION MODEL 


To evaluate the efficacy of the optimized dually polarized data 
communications system, one would require a detailed knowledge of 
the functional and statistical behavior of the complex matrix channel 
characteristics. Since fading events are not very frequent, proper 
complex matrix channel characterization typically requires a few years 
of coherent measurements. Such experiments have not been performed 
to date and only partial data, mainly amplitude characteristics of the 
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in-line channels | Ci,| and | Co2|, have been measured over a limited 
number of propagation paths. 

The propagation model that we shall employ incorporates present 
knowledge with suggested extensions for the phase as well as the cross- 
polarization characteristics. Although this model is incomplete, we 
can still obtain meaningful qualitative results and use them for com- 
parative evaluations of various equalization methods. 

The behavior of | Ci, | and | Co.|, the single channel (in-line) ampli- 
tude characteristics, are well established and documented.”””* To 
review briefly, the generic form of the single channel characteristic is 


2 1 
C(z) = all — b exp(—2zizr/T)]; [z| < 3° 
In the above representation 7 is a fixed parameter (6.3 ns), a is real, 


and b is complex valued, (|b| < 1). The following definitions and 
statistics are usually associated with the parameters in (77), 


(77) 


A = —20 log a 
B = —20 log(1 — [b]) 
o = arg(b). (78) 


The parameters B and ¢ are taken to be independent random variables 
possessing probability densities 


P(B) = x exp(—B/3.8) 


and 
_ J5/6 x, |¢| < 2/2 
pe = {em r/2<|¢| <x° 


The parameter A is a Gaussian random variable with average value 
dependent on B 


(79) 


E[A] = 24.6(500 + B*)/(800 + B*) in dB (80) 


and standard derivation 
oa —5 in dB. (81) 


The single channels can be viewed as being in one of two states with 
associated probabilities, 
Pr(unfaded state) = 0.99689 
Pr(faded state) = 0.00311. . (82) 


The unfaded state is characterized by C(z) = 1. 
The structure of the off-diagonal terms of the matrix C(z), repre- 
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senting the cross-polarization interference, is based upon some exper- 
imental evidence and heuristic arguments:”° °° 


| Cyo(z) | = | RiCu(z) + ReCoo(z) + Rs | 
| Coi(z) | = | RaCir(z) + R5Coo(z) + Rel, (83) 


where k,, ko, k4, and ks are constants that incorporate the nonideal 
properties of the antennas and waveguide feeds at both ends of the 
channel. We assume that dispersion due to multimode conversion, 
propagation, and reflections, in the waveguide feeds, has either been 
eliminated (in the 6-GHz and higher bands) or is nonexistent (in the 
4-GHz band). Typical values of these constants, which vary from one 
hop to another, are in the -35 to —20 dB range. The terms R3 and Re 
are hypothesized to be independent complex Gaussian random varia- 
bles. One may view each of these variables as the result of an inde- 
pendent ray, which would contribute a nondispersive cross-polariza- 
tion response except for a time delay. Typical values of the variance 
or, associated with these Rayleigh random variables, is in the —45 to 
—35 dB range. 

Current data?’ and previous experience” indicate that |C,,| and 
| Co2| are highly correlated, i.e., they practically fade in unison. It has 
been observed in some cases that | C,; | and | C22 | are displaced relative 
to one another across the frequency band. 

Based on the above discussion, we use for our numerical evaluations 
the following extended model (to include amplitude as well as phase 
characteristics) for the dually polarized fading radio channel 


Cy,(z) = af{l — b exp(—2zizr/T)], }z| < 1/2 
C..(z) = a[l — b exp(—2zi(z — Az)r/T)]; 
Az is uniformly distributed over | Az| < AZmax | | (84) 
Cyo(z) = RyCi(z) + koCo2(z) + R3 exp(—2rizD,/T) 
Co1(z) = RyCir(z) + R5Co2(z) + Re exp(—2r1zD2/T) 


As can be seen, we have also associated different time delays, D, and 
Dz, with R3 and Rs. Their values and influence will be discussed in a 
later section. We shall use (84) to evaluate the normalized data rates 
of dually polarized radio channels. 


IX. NUMERICAL SIMULATION STUDIES AND RESULTS 


A meaningful performance measure of a particular communication 
technique is the probability distribution of normalized data rate that 
can be supported at a certain error rate objective. The normalized data 
rate is I = (log.L”)/WT in b/s/Hz, where L? is the total number of 
points in the QAM constellation and WT = 1 in our case. I can be 
obtained from the Chernoff bounds on the probability of error devel- 
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oped in Appendix C. For a fixed error rate we determine the maximum 
number of b/s/Hz supported by the particular channel at a given CNR. 
By computing the maximum number of b/s/Hz supported by each 
member of a large representative population of matrix channels, we 
obtain an estimate of the probability distribution of normalized data 
rates. This distribution is then used to estimate the probability of 
outage. 

More specifically, if F(r) is the probability distribution of normalized 
data rates associated with a particular equalizer/canceler structure 
and an outage objective a is set, then the value r, for which F(r,) = a 
represents the maximum data rate at which it is possible to commu- 
nicate data and meet the set outage objectives. 

We evaluate, via Monte Carlo simulation, the normalized data rates 
that can be supported by our model of a dually polarized radio channel 
employing various equalization/cancellation structures at a prescribed 
error rate. Initial sanity checks for correlated and independent chan- 
nels subjected to flat fading were used in the early stages of the 
program debugging. As we intended to simulate a large number 
(25,000) of fading events, we have utilized the CRAY computer. 

Initial studies were devised to evaluate the sensitivity of the nor- 
malized mean-square errors, MSE,/oz and MSE,,/o3 from (31) and 
(40), to the propagation parameters Azmax, D:/T, and D2/T suggested 
in (84). For a small subset of fading channels, we varied Azmax from 0 
to 0.8 and D,/T and D./T from 0 up to 3. The results indicated 
that the normalized mean-square errors were not very sensitive to 
these parameters when the other propagation parameters satisfy 
{| |} < — 20 dB, 7 = 1, 2, 4, 5 and og < —35 GB. (In the practical case 
of finite tap equalizers, there may be more sensitivity to D,/T and 
D2/T.) In the same calculations we have also observed insensitivity to 
the phases associated with {k;} in (84). 

Based upon these observations, we chose the following worst values 
for our subsequent evaluations 


ky = Ro _ Rg = ks = 0.1; OR = —35 dB. (85) 
Other parameters were chosen as follows: 
D,/T = D2/T = 0.27 and A2max = 0.15. 


We have generated two sets of 25,000 simulated frequency charac- 
teristics, each being obtained via Monte Carlo simulation of the 
random variables A, B, ¢, R3, and Rg, in (78) and (84). One set was 
generated for the case of the cross-polarized radio channels as de- 
scribed in (84). In this case the channels are correlated. The other set 
was generated for the case of a joint fade of two independent channels 
where the A, B, and > parameters of C,, and Cy. in (84) are chosen 
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independently with identical statistics. The interest in this extremal 
and “nonphysical” dual radio channel case is to demonstrate the 
inherent optimization advantages obtained when the two channels are 
independent. 

For given probabilities of errors, P.; and P.2, which we had desig- 
nated as outage thresholds (typically 10~*), we calculated the various 
normalized channels data rates for each simulated frequency charac- 
teristic of the appropriate set. This was accomplished by evaluating 
the individual mean-square errors in (31), (43), (58), and (64) and 
inserting them into (116) and (117). As we pointed out in Appendix 
C, these probabilities are upper bounds since exact expressions are 
not possible to obtain. We thus calculated the individual and total 
normalized channels data rates for the optimized receiver and jointly 
optimized transmitter-receiver equalizer/canceler, as well as their 
Matched Filter (MF) bounds, using these estimates. In addition, we 
also calculated the corresponding Shannon (information theoretic) 
limit from (76). The results are presented in the form of curves of 
outage probability versus normalized data rate, in which we incorpo- 
rated the probabilities of being in a fading state given in (82). In Figs. 
3 and 4, we obtained these curves for various situations, namely, no 
CPI; {k;} = 0 and op = —35 dB; and {k;} = 0.1 and op = —35 dB. Also 
in Figs. 3 and 4 CNR = 63 dB; 7+/T = 0.189 (30-MHz bandwidth); 
D, = Do = 0.27T and Pa = P.. = 1074, 

Figures 3a and b exhibit normalized data-rate distributions for 
correlated channels and for both optimized receiver and jointly opti- 
mized transmitter-receiver, respectively. For the optimized receiver we 
note that, in the absence of CPI (solid curves), there is no significant 
difference between the data rates attained with the equalizer and the 
matched filter bound, about 0.7 b/s/Hz per channel. (When CPI is 
absent and the channels are correlated, the performance—outage 
probability versus normalized data rate—of each individual radio 
channel is identical to that of the single radio channel.** We can, 
therefore, assess the penalties associated with dually polarized radio 
transmission from performance degradation relative to the case when 
CPI is absent.) The corresponding combined channels rates are dis- 
placed by approximately 5 b/s/Hz relative to the Shannon limit. In 
addition, the combined channels rate is twice that of an individual 
channel. When CPI consists of the Rayleigh distributed variables only 
(dashed curves), we note an improvement in the matched filter bound 
and the information theory limit while an individual channel rate with 
the equalizer degrades by about 1.5 b/s/Hz. (This degradation is the 
penalty of dually polarized radio transmission using a linear receiver 
equalizer only. The matched filter bound as well as Shannon’s limit 
indicate that CPI can be used to our advantage rather than detriment.) 
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Fig. 3—(a) Receiver optimized equalizer/canceler. Outage probability versus normalized data rate for correlated channels. (b) Jointly optimized 
transmitter-receiver equalizer/canceler. Outage probability versus normalized data rate for correlated channels. 
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Fig. 4—(a) Receiver optimized equalizer/canceler. Outage probability versus normalized data rate for independent channels. (b) Jointly optimized 
transmitter-receiver equalizer/canceler. Outage probability versus normalized data rate for independent channels. 


Note that the combined channels data rate is greater than twice the 
individual channel rate, as the combined rate is degraded by only 1.5 
to 2.5 b/s/Hz. The physical interpretation of this phenomenon is that 
the independent CPI components R3 and R, provide a certain degree 
of diversity that can be exploited to advantage as in the MF bound 
and the Shannon limit, albeit the equalizer in the optimized linear 
receiver is able to utilize it to a very limited extent. 

When we add to the CPI the additional terms {k;} = —20 dB, all the 
curves are only slightly perturbed. This implies that the contribution 
of the antennas and their feeds can be practically neutralized by the 
equalizer. 

Figure 3b demonstrates the performance of the jointly optimized 
transmitter-receiver for the same set of simulated channel frequency 
characteristics used in Fig. 3a. We note an improvement of 2 to 2.5 
b/s/Hz over Fig. 3a in the matched filter bound in the absence of CPI 
(solid curves) while the corresponding improvement in the combined 
rate with the equalizer is under 1 b/s/Hz. The payoff for joint trans- 
mitter-receiver optimization results when CPI is present (dashed 
curves). As can be seen, about 1 b/s/Hz in individual channel rate and 
2 to 3 b/s/Hz in combined rate are gained over Fig. 3a, in the critical 
region given by probability < 3 x 107°. The differences between the 
rates with and without CPI are much smaller in Fig. 3b, indicating 
barely any penalty for dually polarized transmission with jointly 
optimized equalization. When the data rate is < 8 b/s/Hz, the equalizer 
appears to utilize the CPI to yield a small advantage over the case 
without CPI. We conclude that the joint optimization utilizes the 
diversity, provided by the random components of the CPI, to a much 
greater extent. Note that there is still a 3-b/s/Hz difference relative 
to the matched filter bound in Fig. 3b. Again, the effects of CPI 
contributed by the antennas and their feeds have been eliminated for 
all practical purposes. 

- In Fig. 5 we compare the normalized data rates of the correlated 
channels, under severe CPI, for the optimized receiver and jointly 
optimized transmitter receiver. In this figure CNR = 63 dB; 7/T = 
0.189 (30-MHz bandwidth); {k;} = ~20 dB; or = 35 dB; Di = De = 
0.27T; and P.; = Pe: = 10~*. For an individual channel, an improvement 
of about 1 b/s/Hz can be achieved, in the region of outage probability 
less than 6 X 107°, with joint transmitter-receiver optimization. This 
advantage can amount to 50 percent or more of relative improvement 
when outage probability is < 3 x 107°, and diminishes in significance 
as the outage probability increases. Note that the matched filter 
bounds of the individual channel rates are better by 2 to 2.5 b/s/Hz 
relative to their corresponding equalizer rates for outage probability 
<5 x 10°°. Such a large relative difference indicates that perhaps 
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Fig. 5—Outage probability versus normalized data rate for correlated channels. 


other equalization/cancellation methods and/or Viterbi decoding*! 
could increase the individual rate of the channels. 

The effects of partial diversity, due to the random component of the 
CPI, where the combined channels rate is more than three times the 
individual rate for the lower values of outage probability can again be 
observed. The total combined rate of the jointly optimized transmitter- 
receiver equalizer is lower by about 4 b/s/Hz and 7 b/s/Hz relative to 
the matched filter bound and the Shannon limit, respectively. The 
combined channels rate of the optimized receiver equalizer can be 
lower by as much as 5 b/s/Hz relative to its matched filter bound. 
These facts indicate that an additional improvement may be feasible 
by employing Viterbi decoding.* If higher values of probability of 
outage are acceptable, an optimized receiver equalizer can come rela- 
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tively close in performance to a jointly optimized transmitter-receiver 
equalizer. 

Figures 4a and b exhibit the independent channels normalized data- 
rate distributions for an optimized receiver and jointly optimized 
transmitter-receiver, respectively. In this case there is diversity be- 
tween the channels, and consequently, the combined channels data 
rate is much higher than twice that of an individual channel. 

Again we note that for the optimized receiver there is no significant 
difference between the data rates supported by the equalizer and the 
matched filter bound in the absence of CPI interference. The total 
data rate is about three to four times that of an individual channel. 
There is, however, a 5-b/s/Hz difference between the Shannon limit 
relative to the correlated channels case of Fig. 3. When CPI is present, 
in the form of R3 and Rg only, (84), we notice a slight degradation in 
the equalized individual channel data rate with a correspondingly 
severe degradation in the total channels data rate. This seemingly 
strange phenomenon can be explained as follows: since the channels 
are independent, the probability of a joint bad fade is very low. Thus, 
in the absence of CPI, we expect one channel to be relatively good— 
capable of supporting a very high data rate—while the other channel 
is subject to a severe fade. When CPI is introduced, the data rate of 
the good channel degrades at a much faster pace, with the equalizer 
not being able to stem this pace as effectively as it can equalize the 
bad channel. Note that the contribution of the antennas and their 
feed is more noticeable here. 

In Fig. 4b we show the performance of the jointly optimized trans- 
mitter-receiver for the same set of simulated channel responses as in 
Fig. 4a. We note a 1- to 2-b/s/Hz improvement in the individual 
channel data rate with the equalizer and the matched filter bound in 
the absence of CPI. In this case the total data rate attainable by the 
matched filter bound has improved by about 1 to 1.5 b/s/Hz relative 
to the optimized receiver in the absence of CPI indicated in Fig. 4a. 
On the other hand, the total data rate with the equalizer is slightly 
inferior to the optimized receiver case. This occurs because we have 
minimized the total mean-square error, which is dominated by the bad 
channel in each pair of simulated responses. We thus rob from the 
good channel some power and give it to the bad one. Consequently, 
the data rate of the good channel is decreased by more than the 
improvement obtained from the bad channel. This effect totally dis- 
appears when CPI is present and the superiority of the joint transmit- 
ter-receiver optimization can be observed to substantially counteract 
CPI. 

In Fig. 6 we compare the normalized data rates of the independent 
channels with the optimized receiver and jointly optimized transmit- 
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Fig. 6—Outage probability versus normalized data rate for independent channels. 


ter-receiver equalizers, subject to the same severe CPI as in Fig. 5. In 
Fig. 6 the CNR = 63 dB; +/T = 0.189 (30-MHz bandwidth); {k;} = —20 
dB; op = —35 dB; D; = D, = 0.27T; Per = Peo = 107*. An improvement 
of 1 to 1.5 b/s/Hz can be achieved in the region of outage probability 
<6 xX 10~° with joint optimization. This improvement is comparable 
to the correlated channels case, Fig. 5, although the rates themselves 
are slightly higher in Fig. 6. For the individual channel rates, we note 
again a 2- to 2.5-b/s/Hz difference between the two equalizers and 
their corresponding match filter bound. 

The rate of the combined channels, being much higher than twice 
the rate of the individual channel, demonstrates the advantages that 
can be obtained by utilizing the diversity of the independent channels. 
Operation at a fixed data rate per channel would not utilize this 
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Fig. 7—(a) Receiver optimized equalizer/canceler. Sensitivity to clear air CNR. Outage probability versus normalized 
data rate for correlated channels. (b) Jointly optimized transmitter-receiver equalizer/canceler. Sensitivity to clear air 
CNR. Outage probability versus normalized data rate for correlated channels. 
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Fig. 8—(a) Receiver optimized equalizer/canceler. Sensitivity to threshold error rate—P., = Peo = P.. Outage probability 
versus normalized data rate for correlated channels. (b) Jointly optimized transmitter-receiver equalizer/canceler. Sensitivity 
to threshold error rate—P,, = Peo = P.. Outage probability versus normalized data rate for correlated channels. 
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Fig. 9—(a) Receiver optimized equalizer/canceler. Sensitivity to clear air CNR. Outage probability versus normalized data 
rate for independent channels. (b) Jointly optimized transmitter-receiver equalizer/canceler. Sensitivity to clear air CNR. 
Outage probability versus normalized data rate for independent channels. 
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normalized data rate for independent channels. (b) Jointly optimized transmitter-receiver equalizer/canceler. Sensitivity to threshold 
error rate—P.; = Pez = P.. Outage probability versus normalized data rate for independent channels. 


diversity, and design considerations would be based upon the individ- 
ual channel performance. If we assume that Viterbi decoding can 
practically realize the matched filter bound, then the corresponding 
single-channel curves in Fig. 6 indicate an approximate 1- to 1.5-b/s/ 
Hz advantage for joint optimization. In Fig. 5 the corresponding curves 
for the correlated channels indicate a marginal advantage for joint 
optimization. Operation at variable data rate, where the rate is ad- 
justed according to the states of the channels, can result in a manyfold 
increase of total data rate. Judging again by the combined channels 
matched filter bounds in Figs. 5 and 6, we see no substantial advantage 
in employing joint transmitter-receiver optimization. 

In Figs. 7 and 8 we exhibit the sensitivity of the normalized data 
rates to variations in CNR (clear air carrier-to-noise ratio) and P, 
(threshold error rates) for the correlated radio channels case. In Figs. 
7 and 9 +/T = 0.189; {k;} = —20 dB; cp = —35 dB; D, = D2 = 0.27T; 
and P., = Pe. = 10~*. In Figs. 8 and 10 CNR = 63 dB; 7/T = 0.189; 
{k;} = —20 dB; op = —35 dB; and D, and D, = 0.27T. Figures 9 and 10 
indicate the corresponding sensitivity for the independent channels 
case. For the individual channels, these figures indicate that, irrespec- 
tive of mode of equalization and correlation between channels, a 3-dB 
increase (decrease) in CNR correspond to 1-b/s/Hz increase (decrease) 
in data rate. Also, a ten-fold decrease in P, corresponds to an approx- 
imate 0.3-b/s/Hz decrease in data rate. These statements hold except 
for very low levels of outage probabilities, < 5 x 107°, where less 
sensitivity is observed. These observations are similar to the case of 
single radio channels.” 


X. SUMMARY AND CONCLUSIONS 


We have presented a theory of optimal (in a least-mean-square 
sense) equalization/cancellation in digital data transmission over two 
cross-coupled linear dispersive channels. This research was primarily 
motivated by problems associated with digital data communications 
over dually polarized fading radio channels, and most of our numerical 
work is directed toward this application. 

Viewing the channel as a two-input port, two-output port linear 
network followed by additive noise, we determined the optimum linear 
receiver structure when the transmitted signals are two independent 
quadrature amplitude modulated data waves. The figure of merit used 
in the optimization was the total mean-square error between the 
desired data symbol and the post-filtered signal sample. We found 
that the structure of the optimum filter is comprised of a 2 by 2 matrix 
matched filter followed by a matrix tapped delay line. The tap coeffi- 
cients are described by a sequence of constant matrices. The utility of 
this structure is that it can be approximated by a finite matrix 
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transversal filter whose matrix taps can be adjusted adaptively. For a 
given channel matrix, a transmitter matrix filter and the optimum 
receiver filter, we have derived a closed-form expression for the mini- 
mum attainable total mean-square error as a functional of the trans- 
mitter matrix. This also made it possible to optimize the transmitter 
subject to a total average power constraint 

The extensive numerical work exhibited in the various graphs is 
based on our suggested propagation model. This model is rich enough 
to allow meaningful performance comparisons of the various equali- 
zation/cancellation methods and their ultimate limits. 

Our major conclusions are: 

1. The effects of Cross-Polarization invertoranee (CPI), contributed 
by the antennas and their feeds, can be practically eliminated by the 
linear equalizer/canceler structures discussed in this paper. This may 
have a significant bearing upon the cross-polarization requirements 
placed on the antennas. 

2. In principle, the random components of CPI can provide partial 
diversity reception. They could be exploited to increase the total data 
rates supported by the channels relative to the case where CPI is 
absent. 

3. A jointly optimized transmitter-receiver equalizer/canceler is 
effective when low outage probabilities are required. At relatively high 
outage probabilities it is of marginal benefit. 


4. Over a range from 10~° to 10~° outage probability, the combined 
channels data rate achieved by the jointly optimized transmitter- 
receiver equalizer/canceler is 4 b/s/Hz less than its matched filter 
bound. Similarly, when only the receiver is optimized, the difference 
is 5 b/s/Hz. This observation applies to both correlated and independ- 
ent channels. This significant difference in data rates warrants further 
investigation of other equalization/cancellation methods, such as de- 
cision feedback’® and/or Viterbi decoding.** 

5. In the same range of outage probability as in 4, the combined 
channels data rate associated with the matched filter bounds differ 
from the Shannon limit by about 3 b/s/Hz when jointly optimized 
transmitter-receiver are used and there is an additional loss of 1 b/s/ 
Hz when only the receiver filter is optimized. This observation is again 
valid for both correlated and independent channels. The only way 
known to recoup some of this loss is by channel coding. 

6. Irrespective of equalization methods and correlation between the 
channels, a 3-dB increase (decrease) in CNR corresponds to 1 b/s/Hz 
increase (decrease) in the achievable data rate of an individual channel. 
Also, a ten-fold decrease in error rate corresponds to an approximate 
0.3-b/s/Hz decrease in individual channel data rate. 
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APPENDIX A 
Integral Equation for Optimum Matrix Filter 


To evaluate (16), express trace & explicitly in terms of W,; and hj, i, 
j = 1, 2, the entries of matrices W and H, respectively: 


trace § = 2 — 2Re \J [Wi(—r)hi(t) + Wie(—7)har(7) 
te Woi(—7)hi2(7) + Wal—r)hatr]dr} 


a | [| Warlr) [? + | Wae(z) |? + | Worlr) |? + | Woe(r) |?]dz 


2 





cas | { [Wi(—r) hilt — nT) + Wo(—7)hi(t — nT)\dz 


2 





+), J [Wii(—r)ha(t — nT) + Woi(—7)heo(r — nT) \dz 


n 


2 





cope J [Wio(—7) hilt — nT) + Wol(—7)hio(r — nT)Jdr 


n 





Ps 
+> J [Wi2(—7)hai(t — nT) + Weo(—7)hoo(r — nT)]d7| . (86) 
We proceed as follows. Set the variations 
Ow,, trace § = 0, j=1,2 
and 
Ow,, trace § = 0, i,j = 1, 2, (87) 


where W;;, = Re(W,;) and W;_7 = Im(W;,;). The variations expressed 
in (87) imply that dw trace £ = 0, which is tantamount to equating 


< trace [E(W + «6W)] = 0, (88) 


where W + ¢6W is given in (16a) and the partial derivatives are with 
respect to ¢;, 1 =j = 1, 2. Detailed calculations of 


r) 
—— trace §=0 
0e11 
yield the integral equation, 
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h(t) = o? Wi,(—7) + > hy (7 — nT) 


i [Wi(—z)hilz — nT) + Woy(—z)hiolz — nT)]*dz 
+ > ho(r = nT) 


_[ (Wul-2hat — nT) + Woi(—z)hee(z — nT)]*dz; (89) 
- trace & = 0 
yields 
he(7) = oP Wi(-7) + Y hilt — nT) 
f [Wi2(—z)hir(z — nT) + Woo(—z)hiolz — nT)]*dz 
+ ¥. ha(r — nT) 


_[ (Wal-adhate — nT) + W22(—z)hoo(z — nT)]}*dz; (90) 


) 
=— trace &=0 
0€21 


yields 
hilt) = o Wi(—7) + X hio(t — nT) 
sf [Wii(—z)his(z — nT) + Woi(—2)hio(z — nT)]*dz 


+ » Ago(r — nT) f [ Wis(—z) hei (z = nT) 


+ Wo(—z)hoolz — nT)]*dz; (91) 
and 
0 
ra trace & 
yields 


Rg(7) = 0° W3e(—7) + Y hor — nT) 


{ [ Wie(—z) har (z — nT) + Woe(—z)hiolz — nT)|*dz 
+ oy hoo(r = nT) 


i [Wio(—z)hai(z — nT) + Weo(—z)hoolz — nT)]*dz, (92) 
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where * stands for complex conjugate. Equations (89) through (92) 
can be put into a matrix form: 


dtrace£| (0 0 5 Mee 
de; | — : a Lj= 1, 2 
yields 


ee hi, Bio) ae Leet 
di(r) h3e(7) Wi2(-7) Wee(—7) 


yy i; : oe Yee same sae i 
n Y—o \Wie(—z) Woeo(—z)]\holz — nT) hea(z — nT) 


h(t — nT) hi(r — nT) 
Vhs — nT) hit - nT))’ 


which is exactly eq. (17) in the text. 


APPENDIX B 
Determination of the Matrix Q 


In (39) we give the elements of the diagonal matrix Q in terms of 
the eigenvalues \, and 2, and the Lagrange multiplier }. We shall 
evaluate \ from the constraint of (38) 


1/2 
2p = i” [qir(z) + qo2(z)]dz. (93) 


From (39) it is clear that there may be a situation where A; or Az SA 
over a limited range of the interval |z| < 1/2. Here we shall develop 
an iterative procedure for determining A and the appropriate subinter- 
vals of z over which q, and qz2 = 0. 

In our first iteration, we equate \ = \ to the smallest eigenvalue 
in the interval, i.e., 


\O = Min [Au(z), Ao(z)].- (94) 


lzls5 


This will ensure that q1; and qz2 in (39) are nonnegative. 
Substituting (39) in (93), we obtain the next iterative value of X: 


{ ne dz aie & 

—1/2 : -1/2 Vo 

- ee ee fre = 
-1/2 a -1/2 de 
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x) = (95) 


If AY < ©, then \ = \” and we can now determine qi: and q22 in 
(39). On the other hand, if \% > \™, we have to determine the 
subintervals sp, and sp2 (supports) in |z] < 1/2 over which qj; and qz2 
are respectively nonnegative. We determine these supports from (39). 
The next iterative value of \ is now 


J e+ ls 
cartier an Cees 
sp, VAz sp Vo 


96 
2 a [=] 2 : a 
sp SPp2 re 


1 


2 = 


From \” we obtain the new supports sp, and sp, and continue iterating 
for \® and so on until 


(m) 


1-o< sits 6<1. (97) 


In our subsequent calculations, we used 6 = 0.000001. Typically, we 
obtained \ within two iterations. The highest number of iterations 
that we observed was four. 


APPENDIX C 
Upper Bound on Error Rate and Channel Data Rate 


If U(t) is the overall matrix system impulse response with Fourier 
transform TU(w), then the vector signal sample is 


S(0) = }) U(—nT)A,. (98) 


The sample in “channel a” is 
S(O)a = (dor + ido s)(Utr + iVadly 
+ Y'[Gnr + dn s)(Unr + iU pr) + (bar + ibn »)(Unr + tUz)] 


+ (bo, + ibo (Us: + iU21,) (99) 
and in “channel b” 
S(0)> = (bor + ibo_)(UGr + iUG?r 


+ YD’ (nr + ibn») (Ure + iU? 5) ca (Any + ian») (UR + iUr “z) | 
+ (dor + ao.2)(Udr + iUG2s), (100) 


where A, = (*) and the complex data symbols are 
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Qn = Anr + 1A, x 
and 
bn = Dar + Ibn 7, 


while the complex matrices are denoted 
U,, = (U4, + iU4 A, Lj=1,2 
with ©’ = >. 


n 
n=0 


Channels a and b have real and imaginary parts corresponding to 
the in-phase and quadrature components in QAM systems. Thus, the 
four data signals are expressed as 


Re[S(0)a] = dorUbr — ao» Udly 
+ DY’ [(Gnr Unt — an x Uns) + (Bnr U3h = bas U?,)] 


+ bo-U# — bo» UF 4s), (101) 
Im[S(0)a] = ao. 2Udt + aor-Udls 
+ Y’ [dn UE + dp-UPD) + (bn U2 + bp-U?)9)] 


+ bo» Us + bo Us, (102) 
Re[S(0)s] = bo-Uér — bo» Us > 
5 DY’ [One U7 a bas Us) + (Gn-Ui? — Ans Urs) ] 


+ ao-Ud; — ao ¢ Us ?s, (103) 
and 


Im[S(0)4] = bo.¢ Ube + bo Udy 
+ Yon Ue - bpp U2? 5) oP (An_¢ Ue + On U}?9)] 


+ ao¢ Ud? + ao-Ut >. (104) 


Equations (101) through (104) are the distorted data symbols. In the 
ideal case, when there is no intersymbol and/or cross-polarization 
interference 


Un = Snol (105) 
and, as can be verified, 
Re[S(0)a] = aor, — Im[S(0)a] = aos, 
Re[S(0),] = bo, and Im[S(0),] = bo 
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We now assume that the data symbols {a,,, Qn_v, Dar, Ong} take on 
values from the positive and negative integers, +1 +8 +...+(L-— 1), 
L even. Consequently, we can calculate 





L?-1 
a 


From the noise model described in (3) and (4) in the text we deduce 
that the output noise in channels a and b can be represented, respec- 
tively, as 


E(ai,) = E(ans) = E(b7,) = Elbny) = (106) 


Voa(0) = He : Wii(—7)va(7)dr + i. ? Woai(—7)(7)dz = (107) 


and 


Yop (0) = ie Wio(—7)va(r)dr + {- Wo2(—7)vp(7) dr. 


Thus, the four noise samples added to (101) through (104) are 
Re[voa(0)], Im[voa(0)], Re[vo,(0)], and Im[v,(0)]. They represent four 
zero-mean Gaussian random variables and the respective variances 
can be readily calculated from (107): 


E[Re{vo.(0)}]? = E[Im{yoa(0)}]? = coer 


2 
O0a_7 


= *| { _ (Wale) |? + | Wai(t) ae (108) 


while 
E{Re{vo,(0)}]? = E[Imf{vo,(0)}]? = oor 


N.| {~ 
= sos = mf (| Woo(t) |* + | Wia(t) ae (109) 


In accordance with the definition in (106), which presumes that 
thresholds are set at 0, +2, +4,..., +(Z — 2), an upper bounding on 
the probability of error in each of the four channels can be obtained 
using Chernoff bounding techniques.*”™* 

The resulting bounds are 


(Uo')* 


Pa < exp f — N A 
op (Al Wal” + | Wail*)) tog 


1 
2 
(110) 
Jraure + [UR |) + lay 


4 
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(U3")? 
No a 
= (( | Woe)? + | Wil? )) +> 
ap «| Wael? + | Wial? )) +5 re 
[raver ae ||?) + ust 


where we used the following notation: 


1/2 
(Wi) = J W;;(z)dz, l, J = 1, ya 
-1/2 


Note that since the matrices U, are Hermitian, Uj) and U?? are real 
valued. Also, from (101) through (104) it is clear that the error rate in 
the quadrature component is equal to the in-phase component in each 
of the respective channels. But, clearly, P.; need not equal Py. 

We now relate the denominator of the exponents in (110) and (111) 
to the individual mean-square errors. Utilizing (17a) and (17b) in (15), 
we obtain for the optimum squared error matrix 


f= 04 [ —U,- US+¥U,U; + | Wate) Wi(e\de} (112) 


Since the diagonal entries of £, £1, and £2 are equal to MSE, and 
MSE», respectively, we reduce (110) and (111) with the aid of (18a) to 


(U3)? a 
Pe. < exp |- Ea ~ o4(1 — UB)? = exp j- a (113) 


gee § MAUS _ be 
ee | Boa am raae | A oan 
with 

ay = (1 — &/04)*/[&j/03 — (&/08)7], jf = 1, 2. 


As can be seen, when the mean-square errors are very small we get 
the well-known relationship” 


P. < exp[-1/MSE];_ j = 1, 2. (115) 


One can easily ascertain that in all the formulas previously derived, 
£/o% and £,,/0% are independent of o%. Utilizing (106) in (113) and 
(114), we obtain after some manipulations the normalized data rates 
of the two channels, J, and Ip, 


T, = logo(1 + 1.5:/]1n Pes |) (116) 
I = log (1+ 1.54 /|In P |). (117) 
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These two quantities, sometimes referred to as efficiency indices, 
will be used to compare the performance of the various equalization 
methods. 
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